1. #1
    oilcountry99
    oilcountry99's Avatar SBR PRO
    Join Date: 08-29-10
    Posts: 268
    Betpoints: 1929

    Anyone web scrape with R? Looking for good resources

    Anyone have any good web scrape resources, tutorials, examples for a total illiterate R newbie? Sports related material would be preferred.

    What are your thoughts on web scraping with R?

  2. #2
    Waterstpub87
    Life-Style Arbitrage
    Waterstpub87's Avatar SBR PRO
    Join Date: 09-10-09
    Posts: 2,763
    Betpoints: 71

    I've never thought of R as being particularly good for webscraping. I see more stuff referencing python.
    175 pts

    3-QUESTION
    SBR TRIVIA WINNER 11/23/2017

    BTP
    Week 8
    4-1-0 292 pts


  3. #3
    A4K
    A4K's Avatar SBR PRO
    Join Date: 10-08-12
    Posts: 2,855
    Betpoints: 12

    Just a word of advice.....

    Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.
    Nomination(s):
    This post was nominated 1 time . To view the nominated thread please click here. People who nominated: KVB

    BTP
    Week 11
    4-1-0 509 pts


  4. #4
    jtoler
    jtoler's Avatar Become A Pro!
    Join Date: 12-17-13
    Posts: 18,599
    Betpoints: 384

    R as opposed to S and T? Definitely R.

  5. #5
    xbalto
    xbalto's Avatar Become A Pro!
    Join Date: 10-14-10
    Posts: 105
    Betpoints: 706

    R sucks for this, use Beautiful Soup in python and dump to a csv for R

  6. #6
    HeeeHAWWWW
    HeeeHAWWWW's Avatar Become A Pro!
    Join Date: 06-13-08
    Posts: 5,022
    Betpoints: 12

    Quote Originally Posted by A4K View Post
    Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.
    That's probably a really good suggestion. Scraping is a massive pita - get some Indian phd to do it for peanuts.

  7. #7
    matthewmsturgeon
    matthewmsturgeon's Avatar Become A Pro!
    Join Date: 08-24-17
    Posts: 11
    Betpoints: 102

    Quote Originally Posted by A4K View Post
    Just a word of advice.....

    Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.

    This is intriguing to me. How would you go about finding someone to do this and to describing what you are after. I looked briefly on fiverr, but I didn't want to create an account. Do you just post as request for what you are after and wait for someone to respond. Do you describe the site you want to scrape and indicate that you want it dumped at some interval (or run manually) into a spreadsheet or My SQL DB or something like that? I'm not trying to get you to explain everything that you did, I just feel like this is a concept worthy of more discussion if you have the time. Thanks.

  8. #8
    Bluehorseshoe
    Bluehorseshoe's Avatar SBR PRO
    Join Date: 07-13-06
    Posts: 10,463
    Betpoints: 149

    Quote Originally Posted by A4K View Post
    Just a word of advice.....

    Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.
    I'm looking for someone to scrape sportsbooks for me if anyone is interested.
    175 pts

    3-QUESTION
    SBR TRIVIA WINNER 11/13/2017


  9. #9
    Bsims
    Bsims's Avatar SBR PRO
    Join Date: 02-03-09
    Posts: 247
    Betpoints: 1459

    I still write code in Basic and cannot directly fetch most web pages. For these, I display the page I want and save the page to a file. I then have code that parses the html and builds line files. I then process these.

    Another technique that I've used after displaying the page desired is to right click and Select All, then copy to the clipboard. The version of Basic that I use is capable of retrieving this clipboard data. Then I can build the line files I need.

    The downside of these approaches is that they are static. It requires my intervention. I cannot simulate an API type application.

  10. #10
    hubie69
    I am JJ's Bookie
    hubie69's Avatar Become A Pro!
    Join Date: 09-16-10
    Posts: 1,784
    Betpoints: 31

    Quote Originally Posted by xbalto View Post
    R sucks for this, use Beautiful Soup in python and dump to a csv for R
    This. Exactly this. Beautiful Soup makes life super easy. I use BS4 with a bash wrapper and insert it into a mysql DB.
    Points Awarded:

    redking gave hubie69 1 Betpoint(s) for this post.


Top