1. 05-10-12 02:10 PM #1
    KennyPowers
    Update your status
    KennyPowers's Avatar Become A Pro!
    Join Date: 01-23-12
    Posts: 5
    Betpoints: 42

    Scraping Data from Covers, Creating an NHL Database

    I'm looking for a way to quickly create a database from the NHL information on Covers from the last 5-10 years. For now I guess I am just interested in the scores of each period as well as the moneyline and U/O.

    I do have quite a bit of programming experience, mostly in Java and VB but I haven't done any web-based programming such as scraping or data mining before.

    Any Java libraries you've used to scrape data that might be useful or links to websites with easier information to parse would be greatly appreciated.
    Reply With Quote Reply With Quote
  2. 05-10-12 05:15 PM #2
    TomG
    TomG's Avatar Become A Pro!
    Join Date: 10-29-07
    Posts: 500
    Betpoints: 1

    Don't go back further than the lockout and you might want to skip a year or two after the lockout when teams were still adjusting to changes in strategy. There are enough NHL games in a season where sample size shouldn't be an issue.
    Reply With Quote Reply With Quote
  3. 05-10-12 05:24 PM #3
    AngryPlanets
    AngryPlanets's Avatar Become A Pro!
    Join Date: 01-08-12
    Posts: 31
    Betpoints: 533

    I used WebHarvest for my Java scraping. Easy and effective.
    Reply With Quote Reply With Quote
  4. 05-10-12 07:39 PM #4
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    If you know VB.NET, you can use either a Webbrowser control, Webclient, or the HttpWebRequest class.

    Personally, I wouldn't scrape from Covers. I'd use a combination of several line services and the NHL website.
    Reply With Quote Reply With Quote
  5. 05-11-12 06:59 AM #5
    strixee
    I think, therefore I win
    strixee's Avatar Become A Pro!
    Join Date: 05-31-10
    Posts: 432
    Betpoints: 1

    I do have quite a bit of programming experience, mostly in Java and VB but I haven't done any web-based programming such as scraping or data mining before.
    Data mining has nothing to do with web-based programming
    For scraping I recommend using PHP+MySQL, it's especially useful if you want to use some output accessible online (plus it can run on a shared hosting).

    As MonkeyF0cker said, Covers isn't a good source to scrape from. I'm actually shocked, that I see odds just for 1 side like here covers.com/sports/odds/linehistory.aspx?eventId=47344&sport=nhl ! Each sportsbook has different vig, so you don't know what the other side price was.
    Covers is too US sports oriented service.
    Reply With Quote Reply With Quote
  6. 05-11-12 07:01 AM #6
    mathdotcom
    mathdotcom's Avatar Become A Pro!
    Join Date: 03-24-08
    Posts: 11,689
    Betpoints: 1943

    Quote Originally Posted by strixee View Post
    Data mining has nothing to do with web-based programming
    For scraping I recommend using PHP+MySQL, it's especially useful if you want to use some output accessible online (plus it can run on a shared hosting).

    As MonkeyF0cker said, Covers isn't a good source to scrape from. I'm actually shocked, that I see odds just for 1 side like here covers.com/sports/odds/linehistory.aspx?eventId=47344&sport=nhl ! Each sportsbook has different vig, so you don't know what the other side price was.
    Covers is too US sports oriented service.
    If you look historically where they're getting their numbers you can infer the other side with reasonable accuracy
    Reply With Quote Reply With Quote
  7. 05-11-12 07:25 AM #7
    strixee
    I think, therefore I win
    strixee's Avatar Become A Pro!
    Join Date: 05-31-10
    Posts: 432
    Betpoints: 1

    Here you can get regular season ML odds from Pinnacle since 2009 http://strixee.mysbrforum.com/blog/1...-pinnacle.html
    Nomination(s):
    This post was nominated 3 times . To view the nominated thread please click here. People who nominated: a4u2fear, bullock, and Juret
    Reply With Quote Reply With Quote 3
  8. 05-11-12 03:07 PM #8
    mbs4
    mbs4's Avatar Become A Pro!
    Join Date: 05-14-10
    Posts: 388
    Betpoints: 4546

    Quote Originally Posted by mathdotcom View Post
    If you look historically where they're getting their numbers you can infer the other side with reasonable accuracy
    Where does covers get their numbers from?
    Reply With Quote Reply With Quote
  9. 05-13-12 12:34 PM #9
    KennyPowers
    Update your status
    KennyPowers's Avatar Become A Pro!
    Join Date: 01-23-12
    Posts: 5
    Betpoints: 42

    Thanks everyone for the replies. I will look into WebHarvest and those VB.NET classes. And thanks for the GREAT link strixee, that, along with the scores of each period should definitely start me in the right direction.
    Reply With Quote Reply With Quote
  10. 05-13-12 03:42 PM #10
    Jrod124
    Update your status
    Jrod124's Avatar Become A Pro!
    Join Date: 10-31-09
    Posts: 5,622
    Betpoints: 535

    you will quit within a week
    Reply With Quote Reply With Quote
  11. 05-14-12 01:29 PM #11
    KennyPowers
    Update your status
    KennyPowers's Avatar Become A Pro!
    Join Date: 01-23-12
    Posts: 5
    Betpoints: 42

    How come? You don't think NHL is worth the time? Or you don't think I'm worth the time
    Reply With Quote Reply With Quote
  12. 05-14-12 01:49 PM #12
    durito
    escarabajo negro
    durito's Avatar Become A Pro!
    Join Date: 07-03-06
    Posts: 13,173
    Betpoints: 438

    Quote Originally Posted by KennyPowers View Post
    How come? You don't think NHL is worth the time? Or you don't think I'm worth the time
    Probably doesn't even understand what you are trying to do.
    Reply With Quote Reply With Quote
  13. 06-04-12 06:44 PM #13
    a4u2fear
    TEASE IT
    a4u2fear's Avatar Become A Pro!
    Join Date: 01-29-10
    Posts: 8,147
    Betpoints: 35459

    ive got all odds, results, win streaks going into game, goals for/against going into game for 2011 season.
    Reply With Quote Reply With Quote
  14. 06-04-12 06:45 PM #14
    a4u2fear
    TEASE IT
    a4u2fear's Avatar Become A Pro!
    Join Date: 01-29-10
    Posts: 8,147
    Betpoints: 35459

    Quote Originally Posted by a4u2fear View Post
    ive got all odds, results, win streaks going into game, goals for/against going into game for 2011 season.
    this includes home/away odds, over/under odds.
    Reply With Quote Reply With Quote
  15. 06-24-12 12:48 PM #15
    357vegas
    357vegas's Avatar Become A Pro!
    Join Date: 10-28-11
    Posts: 35
    Betpoints: 0

    Has anyone scraped data from Jeff Sagarin usatoday pitching? I'm looking for a little help to gether data from this site. I have no clue how to, I just know how to use the data.
    Reply With Quote Reply With Quote
  16. 06-29-12 12:37 AM #16
    newbottles
    newbottles's Avatar Become A Pro!
    Join Date: 06-22-12
    Posts: 312
    Betpoints: 475

    Quote Originally Posted by a4u2fear View Post
    this includes home/away odds, over/under odds.
    I am new to this, so pardon me if this is an ignorant question. Is this data from some sort of manual or semi-automated daily collection during the season? Or from an an automated data collection approach?
    Reply With Quote Reply With Quote
  17. 07-07-12 09:27 AM #17
    a4u2fear
    TEASE IT
    a4u2fear's Avatar Become A Pro!
    Join Date: 01-29-10
    Posts: 8,147
    Betpoints: 35459

    Quote Originally Posted by newbottles View Post
    I am new to this, so pardon me if this is an ignorant question. Is this data from some sort of manual or semi-automated daily collection during the season? Or from an an automated data collection approach?
    manually performed it and manipulated. i did however find out that you can do a much easier web query through Excel
    Reply With Quote Reply With Quote
  18. 12-10-12 10:48 PM #18
    degengamble
    degengamble's Avatar Become A Pro!
    Join Date: 12-03-12
    Posts: 39
    Betpoints: 621

    Easiest way to do it is with excel - web queries / vba
    Reply With Quote Reply With Quote
  19. 12-11-12 03:29 AM #19
    allin1
    Update your status
    allin1's Avatar Become A Pro!
    Join Date: 11-07-11
    Posts: 4,555
    Betpoints: 1

    outwit hub is very good for beginners
    Reply With Quote Reply With Quote
Top
« Previous Thread | Handicapper Think Tank | Next Thread »