Anyone web scrape with R? Looking for good resources

[oilcountry99]

Anyone have any good web scrape resources, tutorials, examples for a total illiterate R newbie? Sports related material would be preferred.

What are your thoughts on web scraping with R?

[Waterstpub87]

I've never thought of R as being particularly good for webscraping. I see more stuff referencing python.

[A4K]

Just a word of advice.....

Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.

[jtoler]

R as opposed to S and T? Definitely R.

[xbalto]

R sucks for this, use Beautiful Soup in python and dump to a csv for R

[HeeeHAWWWW]

Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.

That's probably a really good suggestion. Scraping is a massive pita - get some Indian phd to do it for peanuts.

[matthewmsturgeon]

Just a word of advice.....

Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.

This is intriguing to me. How would you go about finding someone to do this and to describing what you are after. I looked briefly on fiverr, but I didn't want to create an account. Do you just post as request for what you are after and wait for someone to respond. Do you describe the site you want to scrape and indicate that you want it dumped at some interval (or run manually) into a spreadsheet or My SQL DB or something like that? I'm not trying to get you to explain everything that you did, I just feel like this is a concept worthy of more discussion if you have the time. Thanks.

[Bluehorseshoe]

Just a word of advice.....

Pay someone on Fiverr.com or post an ad on CraigsList to do the scraping and sorting for you. I did it this way and it cost me $50. The results were far better than I could have managed on my own.

I'm looking for someone to scrape sportsbooks for me if anyone is interested.

[Bsims]

I still write code in Basic and cannot directly fetch most web pages. For these, I display the page I want and save the page to a file. I then have code that parses the html and builds line files. I then process these.

Another technique that I've used after displaying the page desired is to right click and Select All, then copy to the clipboard. The version of Basic that I use is capable of retrieving this clipboard data. Then I can build the line files I need.

The downside of these approaches is that they are static. It requires my intervention. I cannot simulate an API type application.

[hubie69]

R sucks for this, use Beautiful Soup in python and dump to a csv for R

This. Exactly this. Beautiful Soup makes life super easy. I use BS4 with a bash wrapper and insert it into a mysql DB.