An introduction to research

**uva3021** · 11-24-10, 12:30 PM

the site for that sports (mlb.com, nba.com...

**aajreyno** · 12-05-10, 10:13 AM

Incredible Stuff

Thank you dearly for the information!

**Whitefin14** · 12-08-10, 12:51 PM

Great information. I look forward to following this post

**dbDan** · 12-15-10, 08:23 AM

A great thread.

**Pokerjoe** · 02-21-11, 10:55 PM

Originally posted by Jeremy Nguyen

Hello to every One
Do anybody remember what time ? the ball kick off from 2nd between Tenn adn Jacksonvill? Please!! Thank you

One of the all time strangest first and only posts.

**Squared Box** · 02-23-11, 03:56 AM

Originally posted by uva3021

that's because "sht" doesn't exist as a sheet. There is something wrong with your naming conventions

I.E. this is how my range, "NFLTeams" is structured

Arizona2009
Atlanta2009
....
NY+JETS2004
....
SAN+DIEGO2000

Every team from 2009 to 2000 is named in accordance to how they are formatted in the statfox link

Copy the names from a team report page, replace all spaces with a "+", the run the code

This works great. However, I'd like to be able to get it all on a single sheet instead of having each team/year on a separate one. How would I do this?

**sonnyhowitzer** · 02-23-11, 07:08 PM

rob peter to pay paul

**kpoutlaw** · 03-01-11, 01:20 PM

Please don't rail me, but I have absolutely no knowledge of computer programming, except when I took a class in middle school, and little in the way of math skills, but I am fiercely determined to become a skilled handicapper like many of the esteemed in this forum. I didn't even know you could create programs to "scrape" data off of the Internet and program it to run simulations or whatever...absolutely fascinating.. So, where do I begin? Do I just start learning Python as suggested by the OP and go from there? Is that what I need to learn to create programs that can determine the winning probabilitiies of a game and the positive edge I have on each bet? I know that there is much more than statistics and math when it comes to betting and handicapping a game, but I think this is a vital aspect too, and I want in! If anyone has any advice whatsoever besides telling me to go work at 7-11 and to give up gambling forever, it would be greatly appreciated.

Thanks in advance.

**Nookx** · 03-08-11, 11:20 AM

"I know that there is much more than statistics and math when it comes to betting and handicapping" I disagree... Even things like trades, injuries, game time conditions, etc. can be mathematically represented.

**Lopaka** · 03-11-11, 12:50 PM

I was trying to work through the example in the OP, but as I tried to download the mlb ex 1.csv file but the website it is hosted on seems to be down. Does anyone have another link to the file? or something equivalent so that I could learn on? Any help would be GREATLY appreciated.

**mrtomk** · 03-13-11, 05:54 PM

I had the same issue as the post above. If anybody still has the original file would they be able to attach it to a post. Really hope to build on the basics I've learnt about Python so far.

Thanks

**rectangl** · 03-22-11, 02:47 AM

Yeah I've gotten the same problem as the above two people; has anyone found a solution?

**ProphetofProfit** · 03-24-11, 08:04 AM

Does anyone know how to scrape data into Excel using only the html code from the required URLs? My first attempt used 'Data>Import from Web' which took 10 seconds to import, and I have to search through about 110,000 individual pages to find the 25,000 pages I need. I thought that by using the html code I could get the time down a little.

**Maverick22** · 03-24-11, 02:16 PM

Originally posted by ProphetofProfit

Does anyone know how to scrape data into Excel using only the html code from the required URLs? My first attempt used 'Data>Import from Web' which took 10 seconds to import, and I have to search through about 110,000 individual pages to find the 25,000 pages I need. I thought that by using the html code I could get the time down a little.

Sounds like you wanted something for nothing

**ProphetofProfit** · 03-24-11, 06:43 PM

Who doesn't? I never asked for any elaborate response, thought that maybe there was a 'only load html code' button somewhere in Excel. Anyway, turns out the data is inaccurate so ignore my previous post.

**Maverick22** · 03-25-11, 09:16 AM

I dont want something for nothing. You cant trust it

**morganwags** · 03-26-11, 02:24 PM

good info

**Brossy10** · 03-31-11, 02:59 PM

i would crush anyone in nba2k11........... boom

**mrtomk** · 03-31-11, 03:48 PM

Very relevant...

**Tsiggy Fresh** · 04-01-11, 09:37 AM

Does any one have completed data and research exel sheets up

**morganwags** · 04-02-11, 04:29 PM

good info

**uva3021** · 04-13-11, 08:40 PM

Been slacking on my python progamming, but created an excel sheet that grab's mlb lines from Pinnacle everyday at 11 AM, and exports to a database

Everything is automated and runs on a scheduler. I haven't touched it in about 10 days and its smooth sailin, save for a few superfluous database entries. When doing a query its irrelevant however.

The problem arises when there are pitching changes and I'm not around. I guess I could export at intervals to different databases, therefore if there is a pitching change one database will always have the appropriate listed starters. By doing this I can also have the day's opener, mover, and closer for each line.

PM me if you are interested

**demens** · 04-13-11, 09:35 PM

Originally posted by uva3021

Been slacking on my python progamming, but created an excel sheet that grab's mlb lines from Pinnacle everyday at 11 AM, and exports to a database

Everything is automated and runs on a scheduler. I haven't touched it in about 10 days and its smooth sailin, save for a few superfluous database entries. When doing a query its irrelevant however.

The problem arises when there are pitching changes and I'm not around. I guess I could export at intervals to different databases, therefore if there is a pitching change one database will always have the appropriate listed starters. By doing this I can also have the day's opener, mover, and closer for each line.

PM me if you are interested

You have anything like that you use for NBA? I recently created a spreadsheet that does some basic things by itself as well and creates lines for me. Iirc i might have even used a formula from your site if it is your site. Not even close to developing a database just yet, not sure how and dont feel like doing the research since using the online one works fine so far.

**uva3021** · 04-13-11, 09:50 PM

Originally posted by demens

You have anything like that you use for NBA? I recently created a spreadsheet that does some basic things by itself as well and creates lines for me. Iirc i might have even used a formula from your site if it is your site. Not even close to developing a database just yet, not sure how and dont feel like doing the research since using the online one works fine so far.

AN NBA one would be no problem, would just have to change PInnacle XML parameters

If the NBA data was an invasive data extraction from statfox then it probably was my site.

**demens** · 04-13-11, 10:56 PM

Originally posted by uva3021

AN NBA one would be no problem, would just have to change PInnacle XML parameters

If the NBA data was an invasive data extraction from statfox then it probably was my site.

This is the formula i meant (found it by searching old posts. i think you're the one that posted it but i'm not sure)

http://sportsobjective.com/wordpress/?p=3554

This website is for sale! sportsobjective.com is your first and best source for all of the information you’re looking for. From general topics to more of what you would expect to find here, sportsobjective.com has it all. We hope you find what you are searching for!

I would definitely make some use of that spreadsheet your talking about it if you're willing to share, not sure, i know some guys are pretty secretive about their stuff around here. I'm still in the novice phase. PM or here, your call.

**dirk93** · 04-14-11, 10:28 AM

uva, I also have an excel file that grabs pitching matchups and lines from pinnacle. However, ever since the site crashed whenever I refresh my data the only thing that shows up is "You need Javascript and enable cookies" or something like that. Any ideas?

**uva3021** · 04-14-11, 12:23 PM

Originally posted by dirk93

uva, I also have an excel file that grabs pitching matchups and lines from pinnacle. However, ever since the site crashed whenever I refresh my data the only thing that shows up is "You need Javascript and enable cookies" or something like that. Any ideas?

Do you extract through XML? XML is more robust than HTML procedures that are contingent on the format of the website and server issues.

**uva3021** · 04-14-11, 12:28 PM

Originally posted by demens

This is the formula i meant (found it by searching old posts. i think you're the one that posted it but i'm not sure)

http://sportsobjective.com/wordpress/?p=3554

This website is for sale! sportsobjective.com is your first and best source for all of the information you’re looking for. From general topics to more of what you would expect to find here, sportsobjective.com has it all. We hope you find what you are searching for!

I would definitely make some use of that spreadsheet your talking about it if you're willing to share, not sure, i know some guys are pretty secretive about their stuff around here. I'm still in the novice phase. PM or here, your call.

That was one of my better efforts. As far as I know I'm the only person in history to develop a line given the NFL future win total, using the Inverse Pythagorean Formula.

**demens** · 04-14-11, 01:05 PM

Originally posted by uva3021

That was one of my better efforts. As far as I know I'm the only person in history to develop a line given the NFL future win total, using the Inverse Pythagorean Formula.

I actually have an old spreadsheet (NBA PLayoffs) that i'm trying to compare to mine. I dont even remember where i found it but it looks like it might be some of your work too.

The problem i have is that its using user defined functions. One is =LOG5FORM, i managed to find what it is on your site. I see you are using Linear EFF/Win% as the teams Win % in the formula. I have no idea what that is. Quick google search didn't produce anything useful but i will look into it more later. For now i saw that those win% are not much different from the Pyth win % so its not that important imo. I've been using the Gaussian ones in my case.

The question i would like some help with is the 2nd function: =per2line

....ehh, whats the definition?

i'm guessing it coverts % to the line?

p.s.
Also i was wondering (if that is your spreadsheet), how were the results?

**demens** · 04-14-11, 01:11 PM

Originally posted by demens

I see you are using Linear EFF/Win% as the teams Win % in the formula. I have no idea what that is.

Ohh, i see there is a little drop down window that lets you choose which % to use, thats cool.

**uva3021** · 04-14-11, 07:11 PM

yeah that was last year's playoff sheet, i haev to make a new one sigh, playoffs about to start

linear EFF% was the best fit winning percentage for the team compared to the entire league given their efficiency numbers, same with Linear Line %, just using line rather than league (imagine plotting the points on a graph and drawing a line of best fit, then adjust winning percentage to that line)

you can really see some value comparing home/away numbers, which is what the sheet last year did by default

last year it did great, hit a nice 4.5/1 future on boston, and with flat betting finished 20+*

good thing too because i was mediocre to bad in all other sports

**dirk93** · 04-15-11, 09:25 AM

Originally posted by uva3021

Do you extract through XML? XML is more robust than HTML procedures that are contingent on the format of the website and server issues.

I am on a Mac and Excel for Mac doesn't support XML I'm pretty sure. I use HTML right now. Even if Excel did allow XML, I don't know how to go about using it.

**demens** · 04-15-11, 01:12 PM

Originally posted by uva3021

yeah that was last year's playoff sheet, i haev to make a new one sigh, playoffs about to start

linear EFF% was the best fit winning percentage for the team compared to the entire league given their efficiency numbers, same with Linear Line %, just using line rather than league (imagine plotting the points on a graph and drawing a line of best fit, then adjust winning percentage to that line)

you can really see some value comparing home/away numbers, which is what the sheet last year did by default

last year it did great, hit a nice 4.5/1 future on boston, and with flat betting finished 20+*

good thing too because i was mediocre to bad in all other sports

Can you tell me what =per2line is?

**hoyas2007** · 04-16-11, 04:31 PM

UVA,
I am trying to follow your instructions on page 3 of this thread to import NFL data from statfox. I created a list of the 32 teams, highlighted the list, and defined it as "NFLteams." I then ran the macro, but all that happened is that Excel created a bunch of new, blank worksheets, each titled with the team name and the year. How do I get it to actually import the data for each team from each year into the corresponding worksheet? Is there another step that I am missing?

Thanks a lot for the instructions so far - it's really helpful.

**hoyas2007** · 04-16-11, 05:33 PM

When I go line by line through the code, the error comes up when it gets to the url query piece. The error box says "Runtime error 9: subscript out of range." Any ideas what this might refer to?

Thanks