1. 10-08-12 09:02 PM #1
    Justin7
    Justin7's Avatar Become A Pro!
    Join Date: 07-31-06
    Posts: 8,577
    Betpoints: 1506

    Getting CBB play-by-play data

    I've tried scraping a few sites, but my IP eventually gets blocked. Statsheet and ESPN don't want you downloading 5k sets of play-x-play per season.

    Statsheet mentioned that their data provider doesn't allow scraping, or for them to sell it. They won't identify the data provider.

    If you wanted to buy this from a primary source, who has it?
    Reply With Quote Reply With Quote
  2. 10-08-12 09:13 PM #2
    wantitall4moi
    wantitall4moi's Avatar Become A Pro!
    Join Date: 04-17-10
    Posts: 3,063
    Betpoints: 3834

    Tried CBS sports? NCAA website itself?
    Reply With Quote Reply With Quote
  3. 10-08-12 09:13 PM #3
    Chi_archie
    GASPING FOR AIR
    Chi_archie's Avatar SBR PRO
    Join Date: 07-22-08
    Posts: 63,130
    Betpoints: 2368

    I do

    I'm not sure if it is anything before 2010 and its just power conferences

    I can't post it.

    the Mods might give me an infraction
    Reply With Quote Reply With Quote
  4. 10-08-12 09:14 PM #4
    muldoon
    muldoon's Avatar Become A Pro!
    Join Date: 01-04-10
    Posts: 4,397
    Betpoints: 7

    XMLTeam.com
    Reply With Quote Reply With Quote
  5. 10-08-12 09:16 PM #5
    wantitall4moi
    wantitall4moi's Avatar Become A Pro!
    Join Date: 04-17-10
    Posts: 3,063
    Betpoints: 3834

    Quote Originally Posted by muldoon View Post
    XMLTeam.com
    have to pay for that one AFAIK

    EDIT: NVM he said he would buy it.
    Reply With Quote Reply With Quote
  6. 10-08-12 09:19 PM #6
    muldoon
    muldoon's Avatar Become A Pro!
    Join Date: 01-04-10
    Posts: 4,397
    Betpoints: 7

    Quote Originally Posted by wantitall4moi View Post
    have to pay for that one AFAIK
    OP said "If you wanted to buy this from a primary source, who has it?"

    Hard to find anything real time that's decent without paying something for it.
    Reply With Quote Reply With Quote
  7. 10-08-12 09:23 PM #7
    wantitall4moi
    wantitall4moi's Avatar Become A Pro!
    Join Date: 04-17-10
    Posts: 3,063
    Betpoints: 3834

    Quote Originally Posted by muldoon View Post
    OP said "If you wanted to buy this from a primary source, who has it?"

    Hard to find anything real time that's decent without paying something for it.
    yeah i edited it after i read the last line.

    I am sure CBSsports would work for awhile. I dont do full play by play scrapes ( no wonder he gets blocked) but 'normal' stuff n problems.
    Reply With Quote Reply With Quote
  8. 10-09-12 12:32 AM #8
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    I do.

    $12k.
    Reply With Quote Reply With Quote
  9. 10-09-12 01:20 AM #9
    rm18
    Update your status
    rm18's Avatar Become A Pro!
    Join Date: 09-20-05
    Posts: 22,291
    Betpoints: 207814

    you can get a vpn with like 100 ips might be cheaper
    Reply With Quote Reply With Quote
  10. 10-09-12 01:25 AM #10
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    I've been scraping from ESPN for years and I've never been IP restricted...
    Reply With Quote Reply With Quote
  11. 10-09-12 01:31 AM #11
    Ace_of_Spades
    Golazo De Riverrrr!
    Ace_of_Spades's Avatar Become A Pro!
    Join Date: 10-14-09
    Posts: 13,518
    Betpoints: 79

    Quote Originally Posted by Chi_archie View Post
    I do

    I'm not sure if it is anything before 2010 and its just power conferences

    I can't post it.

    the Mods might give me an infraction
    You are a joke.
    Reply With Quote Reply With Quote
  12. 10-09-12 05:26 AM #12
    jjgold
    jjgold's Avatar Become A Pro!
    Join Date: 07-20-05
    Posts: 388,190
    Betpoints: 10

    It will cost you about $800 a month through stats.com or even maybe sportsnetwork
    Reply With Quote Reply With Quote
  13. 10-09-12 05:27 AM #13
    jjgold
    jjgold's Avatar Become A Pro!
    Join Date: 07-20-05
    Posts: 388,190
    Betpoints: 10

    http://www.stats.com/

    http://www.sportsnetworkdata.com/
    Reply With Quote Reply With Quote
  14. 10-09-12 06:27 AM #14
    Emily_Haines
    Emily_Haines's Avatar Become A Pro!
    Join Date: 04-14-09
    Posts: 15,887
    Betpoints: 15307

    Each university has their own site.

    http://www.hawkeyesports.com/sports/....html#GAME.PLY

    You can easily scrap a seasons worth of games and probably much more before getting blocked. Once your done just move onto the next one. Lot's of them use the same format, so it should not be that bad of a programming nightmare.
    Reply With Quote Reply With Quote
  15. 10-09-12 06:34 AM #15
    jjgold
    jjgold's Avatar Become A Pro!
    Join Date: 07-20-05
    Posts: 388,190
    Betpoints: 10

    Pro Players Pay for it

    Reliable and in programming format
    Reply With Quote Reply With Quote
  16. 10-09-12 12:17 PM #16
    rsigley
    rsigley's Avatar Become A Pro!
    Join Date: 02-23-08
    Posts: 304
    Betpoints: 186

    since i have it set up to parse data all day fro m espn.com i took an old box i never used and for a test i just parsed every pbp on espn for cbb no ip ban

    so i deleted the db and did it again no ip ban

    so i deleted the db and did it again no ip ban

    then i gave up. so for 4 non stop hours i pulled data from espn.com hundreds of thousands of pbp/box score and no ip ban
    Reply With Quote Reply With Quote
  17. 10-09-12 12:21 PM #17
    jjgold
    jjgold's Avatar Become A Pro!
    Join Date: 07-20-05
    Posts: 388,190
    Betpoints: 10

    depends what frequency rate is

    Every few minutes and you might be ok

    every second and could have problems
    Reply With Quote Reply With Quote
  18. 10-09-12 02:04 PM #18
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Quote Originally Posted by rsigley View Post
    since i have it set up to parse data all day fro m espn.com i took an old box i never used and for a test i just parsed every pbp on espn for cbb no ip ban

    so i deleted the db and did it again no ip ban

    so i deleted the db and did it again no ip ban

    then i gave up. so for 4 non stop hours i pulled data from espn.com hundreds of thousands of pbp/box score and no ip ban
    LOL. Yup. I just did the same.

    Only once though.

    Never been IP restricted from there.
    Reply With Quote Reply With Quote
  19. 10-09-12 02:07 PM #19
    Maverick22
    Maverick22's Avatar Become A Pro!
    Join Date: 04-10-10
    Posts: 807
    Betpoints: 58

    Amazon cloud computing. You are welcome.
    Reply With Quote Reply With Quote
  20. 10-09-12 02:11 PM #20
    Justin3587
    Justin3587's Avatar Become A Pro!
    Join Date: 08-29-12
    Posts: 2,566
    Betpoints: 450

    What is scraping?
    Reply With Quote Reply With Quote
  21. 10-09-12 02:13 PM #21
    Maverick22
    Maverick22's Avatar Become A Pro!
    Join Date: 04-10-10
    Posts: 807
    Betpoints: 58

    Quote Originally Posted by Justin3587 View Post
    What is scraping?
    To take the data from a website and store it off somewhere else. Without actually having direct access to the websites data
    Reply With Quote Reply With Quote
  22. 10-09-12 02:20 PM #22
    Justin3587
    Justin3587's Avatar Become A Pro!
    Join Date: 08-29-12
    Posts: 2,566
    Betpoints: 450

    Quote Originally Posted by Maverick22 View Post
    To take the data from a website and store it off somewhere else. Without actually having direct access to the websites data
    I see, this may be a dumb question, but what is the point if you can log onto the sites and just view it whenever?
    Reply With Quote Reply With Quote
  23. 10-09-12 02:55 PM #23
    jgilmartin
    jgilmartin's Avatar Become A Pro!
    Join Date: 03-31-09
    Posts: 1,119
    Betpoints: 7

    Quote Originally Posted by Justin3587 View Post
    I see, this may be a dumb question, but what is the point if you can log onto the sites and just view it whenever?
    You can make your own database and run your own analysis on it that wouldn't be possible via simply viewing it on the site.
    Points Awarded:

    Maverick22 gave jgilmartin 2 SBR Point(s) for this post.

    Reply With Quote Reply With Quote
  24. 10-09-12 05:22 PM #24
    rsigley
    rsigley's Avatar Become A Pro!
    Join Date: 02-23-08
    Posts: 304
    Betpoints: 186

    Quote Originally Posted by jjgold View Post
    depends what frequency rate is

    Every few minutes and you might be ok

    every second and could have problems
    no pause
    Reply With Quote Reply With Quote
  25. 10-09-12 05:27 PM #25
    muldoon
    muldoon's Avatar Become A Pro!
    Join Date: 01-04-10
    Posts: 4,397
    Betpoints: 7

    Quote Originally Posted by Justin3587 View Post
    I see, this may be a dumb question, but what is the point if you can log onto the sites and just view it whenever?
    Some may want to store the data to use later (or to analyse in real time)

    Some may want to relay that info to their own site or app.
    Reply With Quote Reply With Quote
  26. 10-09-12 05:33 PM #26
    339955
    339955's Avatar Become A Pro!
    Join Date: 07-20-12
    Posts: 198
    Betpoints: 0

    rsigley how many bets do you make a day how do you have time to make them all? what sites do you et on?
    Reply With Quote Reply With Quote
  27. 10-10-12 07:40 PM #27
    Miz
    Miz's Avatar Become A Pro!
    Join Date: 08-30-09
    Posts: 695
    Betpoints: 3162

    You can use python to change what type of browser the site thinks you're using, space out your queries, goof w IPs, etc.

    We've had good luck doing that.
    Reply With Quote Reply With Quote
  28. 10-10-12 11:17 PM #28
    Justin7
    Justin7's Avatar Become A Pro!
    Join Date: 07-31-06
    Posts: 8,577
    Betpoints: 1506

    Thanks, I got what I needed by rotating through proxies.
    Reply With Quote Reply With Quote
  29. 10-10-12 11:34 PM #29
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Quote Originally Posted by Justin7 View Post
    Thanks, I got what I needed by rotating through proxies.
    Reply With Quote Reply With Quote
  30. 10-11-12 12:26 AM #30
    FreeFall
    FreeFall's Avatar Become A Pro!
    Join Date: 02-20-08
    Posts: 3,365
    Betpoints: 2391

    Quote Originally Posted by MonkeyF0cker View Post
    HEY Your're a fvckng liar and a monkey rooster_eater. YOU NEED TO ROTATE THOSE PROXISSES BRO! SEPN bans every user from viewing their site b/c it helps them with ad revenue. God do you not read the thread?
    Reply With Quote Reply With Quote
  31. 10-11-12 06:59 PM #31
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990
    Betpoints: 1

    Use a proxy.
    Reply With Quote Reply With Quote
  32. 10-11-12 07:36 PM #32
    tony_come
    tony_come's Avatar Become A Pro!
    Join Date: 03-31-10
    Posts: 21,695
    Betpoints: 1122

    use excel sheets
    Reply With Quote Reply With Quote
Top
« Previous Thread | Handicapper Think Tank | Next Thread »