1. #1
    Bluehorseshoe
    Bluehorseshoe's Avatar Become A Pro!
    Join Date: 07-13-06
    Posts: 14,936
    Betpoints: 1551

    So who can I hire for scraping sites.

    Hit me up in a pm.

  2. #2
    shadymcgrady
    shadymcgrady's Avatar Become A Pro!
    Join Date: 02-27-12
    Posts: 10,036
    Betpoints: 2978

    Just download and customize a webscraper

  3. #3
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    Where from?

  4. #4
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    What are you looking to scrape?

  5. #5
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    daily odds nhl mlb, used to use VI site with excel

  6. #6
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    Quote Originally Posted by oilcountry99 View Post
    daily odds nhl mlb, used to use VI site with excel
    if i just use get data from web this loads pretty cleanly. Is it that you don't know turn that into a macro?

  7. #7
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    They no longer show the sportsbooks I need, i need a new source to pull into excel.

  8. #8
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    Quote Originally Posted by oilcountry99 View Post
    They no longer show the sportsbooks I need, i need a new source to pull into excel.
    If you find a source, post and we can see how easy it is to scrape. I've started to try to do sbrodds with python, but it also seems like it might just be easier with excel.

  9. #9
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    I would like to use SBRodds as well, I haven't had success scraping it with excel, but I"m far from an expert in excel. I don't know how to use python, I've dabbled with it but that's about it.

    how about c o v e r s??

  10. #10
    turtledoves
    turtledoves's Avatar Become A Pro!
    Join Date: 08-27-17
    Posts: 3,398
    Betpoints: 4603

    what's a working odds scraper program like this worth to you hypothetically?

    you can look on freelance programming websites to hire someone with test automation/selenium skills. testers with these skills should probably able to scrape most sites. coding to verify a website and collect data from a website can be very similar, that's why you might be able to hire a tester cheaper than a software developer.

    code quality can be shitty though and you might a get a mess that is harder to maintain.

    if the data source/site changes, you might have to hire again for touchups/fixing. you should probably have multiple data sources in case one goes down/changes.

    i bet plenty of gamblers here have working scrapers of what you need. you can probably make an offer if you find the right person...

  11. #11
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

  12. #12
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    Waterstpub87......I've used that site in the past and didn't really care for it. I'd need something comparable to vi. i like the sbr format. Appreciate your help.

  13. #13
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    https://www.oddsshark.com/mlb/odds
    https://www.oddsmarket.com/baseball/

    I would use python if I had the code and it output to .csv, not ideal but it would work. I prefer to keep things in excel if possible.

  14. #14
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    I'm also not looking for historical odds, just the current day, including the opener. Does that make things easier.

    Sorry for hijacking the OP's thread, but its on topic so figured it was ok to post here

    SBR should just add an export to .csv button on the odds page. I'm going to go suggest that right now

  15. #15
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    Quote Originally Posted by oilcountry99 View Post
    I'm also not looking for historical odds, just the current day, including the opener. Does that make things easier.

    Sorry for hijacking the OP's thread, but its on topic so figured it was ok to post here

    SBR should just add an export to .csv button on the odds page. I'm going to go suggest that right now
    Yes, but not by much. I also might keep it in excel. I am trying to scrape historical college basketball lines at the moment. Let you know if I get something working.

  16. #16
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    Quote Originally Posted by Waterstpub87 View Post
    Yes, but not by much. I also might keep it in excel. I am trying to scrape historical college basketball lines at the moment. Let you know if I get something working.
    I hope you can come up with something, good luck!

  17. #17
    Gaze73
    Gaze73's Avatar Become A Pro!
    Join Date: 01-27-14
    Posts: 3,105
    Betpoints: 1192

    My brother can do that sort of thing, he made me a scraper.

  18. #18

  19. #19
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    Quote Originally Posted by Bsims View Post
    Appreciate the link. I found it, there was some error with it that I found, I forget what it was. Unfortunately, Oilcountry seems to be looking more for a day to day click a button get todays lines in a spreadsheet, rather than an odds archive.

    Too much programming and shit to do for sports betting. August - April the busiest time. So much in the backlog.

  20. #20
    Bsims
    Bsims's Avatar Become A Pro!
    Join Date: 02-03-09
    Posts: 827
    Betpoints: 13

    Quote Originally Posted by Waterstpub87 View Post
    Appreciate the link. I found it, there was some error with it that I found, I forget what it was.
    There are errors in the archive data. It also isn't easy to use. I wrote a program to edit these lines and drop the bad ones .

  21. #21
    Bsims
    Bsims's Avatar Become A Pro!
    Join Date: 02-03-09
    Posts: 827
    Betpoints: 13

    Quote Originally Posted by Waterstpub87 View Post
    Unfortunately, Oilcountry seems to be looking more for a day to day click a button get todays lines in a spreadsheet, rather than an odds archive.

    Too much programming and shit to do for sports betting. August - April the busiest time. So much in the backlog.
    I don't directly scrape web pages, the language I use doesn't really have the tools necessary. I save the web pages I want. I then have code that can read this file and create a comma delimited file in spreadsheet format. Figuring out where the actual data is in the web page is difficult but necessary. Periodically a site will make changes to there web pages thus requiring an effort to modify the code.

    You're right about the work involved.

  22. #22
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    Quote Originally Posted by Bsims View Post
    I don't directly scrape web pages, the language I use doesn't really have the tools necessary. I save the web pages I want. I then have code that can read this file and create a comma delimited file in spreadsheet format. Figuring out where the actual data is in the web page is difficult but necessary. Periodically a site will make changes to there web pages thus requiring an effort to modify the code.

    You're right about the work involved.
    What language are you using?

  23. #23
    Bsims
    Bsims's Avatar Become A Pro!
    Join Date: 02-03-09
    Posts: 827
    Betpoints: 13

    Quote Originally Posted by Waterstpub87 View Post
    What language are you using?
    I write in Basic. Almost all my code is in Power Basic. It is a compiler/interpreter. I cannot download any complex web pages in it. So I use the Liberty Basic compiler to download most web pages I want. I use data files to request the pages I want and the Liberty Basic code fetches them and saves them. Then the Power Basic code processes the saved pages.

    I'd really like to learn Python and be able to combine the web fetching and data processing in one place. Then I could write some API's to really automate some processing. Unfortunately my priority now is bitcoin analysis looking for buy and sell price points.

  24. #24
    Nappyx
    Nappyx's Avatar Become A Pro!
    Join Date: 11-05-17
    Posts: 652
    Betpoints: 616

    Message me about what you want and I may be able to help. VB.Net coder that has scraped many sites.

  25. #25
    sonnyboy
    sonnyboy's Avatar Become A Pro!
    Join Date: 08-10-19
    Posts: 2
    Betpoints: 42

    download the SBR daily web page/table to excel

    Can the SBR daily web page/table be downloaded.
    I made a macro with the macro recorder to do that with the DonBest site , but no matter what i try i cant download the SBR site into excel.
    An earlier poster said that he was going to suggest that they put a link to download it into excel. I wonder if that has been asked.
    sonnyboy

  26. #26
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    Quote Originally Posted by sonnyboy View Post
    Can the SBR daily web page/table be downloaded.
    I made a macro with the macro recorder to do that with the DonBest site , but no matter what i try i cant download the SBR site into excel.
    An earlier poster said that he was going to suggest that they put a link to download it into excel. I wonder if that has been asked.
    sonnyboy
    I asked ... and of course not got a response.

  27. #27
    turbobets
    turbobets's Avatar Become A Pro!
    Join Date: 01-13-06
    Posts: 923
    Betpoints: 13715

    Quote Originally Posted by sonnyboy View Post
    An earlier poster said that he was going to suggest that they put a link to download it into excel. I wonder if that has been asked.
    sonnyboy
    They could use it as an incentive to get posters to sign up for a pro membership and it would be worth it.

  28. #28
    wadesi
    wadesi's Avatar Become A Pro!
    Join Date: 09-01-12
    Posts: 15
    Betpoints: 842

    I know this is a old thread but has anyone come up with a way to scrape the sbrodds site into excel or csv? I would def pay for a pro membership if this functionality existed....

  29. #29
    oilcountry99
    oilcountry99's Avatar Become A Pro!
    Join Date: 08-29-10
    Posts: 707
    Betpoints: 1094

    check out octoparse....works well

  30. #30
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    Sub scrapegames()
    Range("A").ClearContents
    Dim Day As String

    Day = Range("K1").Value

    Application.CutCopyMode = False
    With ActiveSheet.QueryTables.Add(Connection:= _
    "URL;https://classic.sportsbookreview.com/betting-odds/nba-basketball/?date=" + Day _
    , Destination:=Range("$A$1"))

    .Name = "?date=20181003"
    .FieldNames = True
    .RowNumbers = False
    .FillAdjacentFormulas = False
    .PreserveFormatting = True
    .RefreshOnFileOpen = False
    .BackgroundQuery = True
    .RefreshStyle = xlOverwriteCells
    .SavePassword = False
    .SaveData = True
    .AdjustColumnWidth = True
    .RefreshPeriod = 0
    .WebSelectionType = xlEntirePage
    .WebFormatting = xlWebFormattingNone
    .WebPreFormattedTextToColumns = True
    .WebConsecutiveDelimitersAsOne = True
    .WebSingleBlockTextImport = False
    .WebDisableDateRecognition = False
    .WebDisableRedirections = False
    .Refresh BackgroundQuery:=False
    End With
    Range("D1").Select
    Application.CutCopyMode = False
    With ActiveSheet.QueryTables.Add(Connection:= _
    "URL;https://classic.sportsbookreview.com/betting-odds/nba-basketball/totals/?date=" + Day _
    , Destination:=Range("$C$1"))

    .Name = "?date=20181003_1"
    .FieldNames = True
    .RowNumbers = False
    .FillAdjacentFormulas = False
    .PreserveFormatting = True
    .RefreshOnFileOpen = False
    .BackgroundQuery = True
    .RefreshStyle = xlOverwriteCells
    .SavePassword = False
    .SaveData = True
    .AdjustColumnWidth = True
    .RefreshPeriod = 0
    .WebSelectionType = xlEntirePage
    .WebFormatting = xlWebFormattingNone
    .WebPreFormattedTextToColumns = True
    .WebConsecutiveDelimitersAsOne = True
    .WebSingleBlockTextImport = False
    .WebDisableDateRecognition = False
    .WebDisableRedirections = False
    .Refresh BackgroundQuery:=False
    End With
    Columns("A:F").Select
    Selection.Replace What:="top", Replacement:="", LookAt:=xlPart, _
    SearchOrder:=xlByRows, MatchCase:=False, SearchFormat:=False, _
    ReplaceFormat:=False
    Selection.Replace What:="Help", Replacement:="", LookAt:=xlPart, _
    SearchOrder:=xlByRows, MatchCase:=False, SearchFormat:=False, _
    ReplaceFormat:=False
    Columns("A:C").Select
    Selection.Replace What:="=", Replacement:="", LookAt:=xlPart, _
    SearchOrder:=xlByRows, MatchCase:=False, SearchFormat:=False, _
    ReplaceFormat:=False


    End Sub

  31. #31
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    the smiley face is A : D

  32. #32
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    you then have to match formulas to put it into a table form.

    This will put spreads in "A" and totals in "C"

  33. #33
    Waterstpub87
    Slan go foill
    Waterstpub87's Avatar Become A Pro!
    Join Date: 09-09-09
    Posts: 4,043
    Betpoints: 7236

    You have to put a date in "k1", like "3/11/2021"

Top