An introduction to research

[Poogs]

ladies ladies, youre killing an otherwise great thread.

[MonkeyF0cker]

I really don't think the discussion we had is too far off topic. The thread is about programmatically scraping data. There really needs to be some concern by those who wish to pursue that venture on how that data is stored, organized, and accessed. CSV's simply aren't practical for most profitable modeling applications.

[sharpcat]

Ijump12,

Please continue with your thread and do not let Monkey destroy it, which has clearly been his intention since day 1

This is a great thread for those of us interested in the subject and your thread has inspired me to purchase a few books to help make this a little more understanding for me. The subject has tweaked my interest for a while now and your thread has helped to point me in the right direction to get started, I also have interest in programming outside of gambling but this is a great starting point to give me the motivation to learn.

[LT Profits]

Thread is also cleaned up now, so carry on Ijump.

[Daniel]

How many of you analytical minds use historical (i.e. last year's) data? I'm guessing you're aware of the Lahman-database @ http://www.baseball1.com/?

There is similar data available for hockey in case people are interested.

[hockeyman30304]

[frankzig]

anybody up and running?

[Meestermike]

How many of you analytical minds use historical (i.e. last year's) data? I'm guessing you're aware of the Lahman-database @ http://www.baseball1.com/?

There is similar data available for hockey in case people are interested.

Thanks Daniel. Any idea on the hockey DB addy?

[Daniel]

Thanks Daniel. Any idea on the hockey DB addy?

The hockey stats are only available if you're a member of a yahoo group (free). The data is also CSV-only and updated after each season, but contains just about every NHL-season's worth of individual and team stats.

It's managed by a single guy, but he appreciates comments and discussion on the yahoo-group's mailing list.

http://sports.groups.yahoo.com/group/hockey-databank/

[Meestermike]

The hockey stats are only available if you're a member of a yahoo group (free). The data is also CSV-only and updated after each season, but contains just about every NHL-season's worth of individual and team stats.

It's managed by a single guy, but he appreciates comments and discussion on the yahoo-group's mailing list.

http://sports.groups.yahoo.com/group/hockey-databank/

Thanks Daniel. Much appreciated

[Nuggz]

Wow I've been learning Python recently and didn't really consider it as a tool for investing in sports. Very cool..

[dinaro7]

i agree

[kisado]

Awesome thread, I think! I'm not computer program savvy enough to fully comprehend what you have posted, but I can tell it's prolly some good shiit. I'm more of a 'feel' bettor if there's such a thing. I do handicap, but I use that data combined with my knowledge of the particular sport and other factors like motivation, past results, etc.

[jbrent95]

Thanks for the introduction to Python.

[Reno Gambler]

This is great stuff for us beginners!!

[frankzig]

very intresting

[CaptainPrice]

importing odds sounds like a pain still

[Wrecktangle]

importing odds sounds like a pain still

if it were easy, we'd all be swilling Mai-tais with the sports groupies in the Bellagio penthouse like Monkey. Instead we seek a zen-like oneness with our data bases ignoring all the worldly pleasures.

[Maverick22]

I have a WELL tested Java Application/Scraper that gets the odds from SBROdds. If someone wants to propose a trade of something worth having...I'd be willing to barter it off...

If anyone is interested, PM me...

[sharpcat]

Greek Sports Picks is now 28-5 in their "MLB Lock of the Day." Today, their pick was the Over of 9 1/2 in the Rockies and Brewers game. It easily covered with 15 runs scored combined between the 2 teams. Try their free pick for tomorrow by dialing their toll free number: 1 - 888 - 608 - 7487. Their free picks record is 14-5. TRY IT NOW! BREAK THE BOOKIES!!! ITS EASY MONEY!

Handicappers digest says these guys are on a 1- 20 run

[yanky yank]

Brazil kicking some chilean butt

[pats3peat]

I'm always struck by how hard it can be to express yourself in print, and the fact that we all use differing terms to label the same items. I'm not a data base guy but data dictionary "thingies" are important even in my simplistic world. I would like to see us stay away from the Players Talk way of solving differences of opinion here in the Tank, however.

I keep saying this to no observable progress: I'd like to see a group form where the interest is sharing checked out data sets. I spend way too much time cross checking data and way too little time on model building and analysis; especially the analysis.

yea i think pretty much everyone wastes way too much time instead of just getting started on a model andend up not even trying. not saying research is a waste of time though

What do you mean, cross-checking each others data sets... to speed up the process? What do you use, mostly just excel?

[Wrecktangle]

yea i think pretty much everyone wastes way too much time instead of just getting started on a model andend up not even trying. not saying research is a waste of time though

What do you mean, cross-checking each others data sets... to speed up the process? What do you use, mostly just excel?

Building fully checked out dbs with all the conditionals (everything that is knowable about a game/team) is damn difficult. Once you have that, model building is relatively easy, and certainly more fun.

[bztips]

These "how-to" posts by lj are by far the most useful things I've seen on this entire site (admittedly I'm relative newbie who has been scouring around for just a little while).

Following his examples, I am now up and running automated daily scrapes of MLB data from covers, baseballprospectus and statcorner. For my purposes, the csv exports are perfect because they can be appended automatically into my databases.

Now I just have to be patient, wait for my sample sizes to grow -- and then it's off to modeling!

[RaginCajun]

i really just got started in handicapping and never thought to do any of this. are u guys making your dbs by using these programs that u made? and what kind of data are you taking from each sport? its ok if u cant reveal this information

[Wrecktangle]

i really just got started in handicapping and never thought to do any of this. are u guys making your dbs by using these programs that u made? and what kind of data are you taking from each sport? its ok if u cant reveal this information

Everything you think might be relevant and maybe even some that might not. Then test them against the lines you scrape. These things can get very large. No one said this is easy.

[Indecent]

Python programmers might want to look into Scrapy. Never used it, but seems like it would be useful for betting purposes.

Upon further reflection.. From the perspective of teaching new programmers, it might be too big of a stretch to teach Python and XPath (and maybe html) all at once. This could be useful for more experienced users, but maybe not quite appropriate for this type of tutorial/userbase.

[Maverick22]

i really just got started in handicapping and never thought to do any of this. are u guys making your dbs by using these programs that u made? and what kind of data are you taking from each sport? its ok if u cant reveal this information

Not sure how everyone else does, so i speak for myself.

I have a program that will go out to the internet stat websites ie CBS Sports. This program gets all the game data available. I take EVERYTHING i can find. And then i take this data and load it into a database. Later, once i have all the data, I figure out what i actually need to use...

PS. Getting ONLY the data you need is a bad practice and will hurt you in the future. And it's lazy. Don't listen to anyone that says "your database should only contain data you plan to use". This is bad when over time your approach changes and you need more (and increasingly specific) data.

[roasthawg]

I've been using PHP with xpath to scrape lately... there's a bunch of different ways though and not all of them involve programming. The hard part is finding that data... not scraping it.

[RaginCajun]

i tried to use the elif function in the second "tutorial" and it said there is a syntax error. i checked and i have everything exactly how it is posted. please help

[CrimsonQueen]

Ijump, this is the best thread ever. Please continue it!!

[eldoctord]

Excellent information. I wonder how many people put this much thought and energy into their decisions. Most of us are impulsive, gut bettors. More power to those who can come up with an edge and consistently win money.

[CrimsonQueen]

your thread has inspired me to purchase a few books to help make this a little more understanding for me.

sharpcat or anyone else: what books are you reading that are about this? that specifically talks about programming databases and modeling for sportsbetting?

I've always had some basic knowledge of programming from various programs, but this has very much grabbed my attention and I would love to delve fully into the world of programming and databases.

[Maverick22]

http://www.amazon.com/Baseball-Hacks-Analyzing-Winning-Statistics/dp/0596009429

[CrimsonQueen]

Thanks maverick22, but that's only about baseball. If the book is sport specific, I'd want it about NCAAB.