1. #1
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    Using Excel to import and filter all Pinnacle lines by sport/league

    uva3021's work on grabbing Pinnacle's XML feed with Excel gave me an idea to take this a step further by building a workbook that allows the user to create a table with odds for any sport/league that Pinnacle offers odds for and displaying it in a customized format e.g. depending on the sport or event. Since Pinnacle explicitly allows anyone to access their XML feed once every 60 seconds, I figured too much harm can't be done by sharing this. Also, if you were to change the settings to import it more often, I am pretty sure your IP would eventually get blocked by Pinny. Obviously there are ways around that too but my point is that as long as everything in the workbook is left the way I set it up, no rules (explicit or implicit) are likely to be broken.

    My goal was to get a working solution, not the prettiest, and in its current version it works for me at least and maybe it can be useful for someone else.

    I wouldn't advice using the code I created as a proper way of writing VBA since my focus was on getting something working quickly. When that failed and the task got infinitely more complicated than I envisioned initially, instead of re-writing much of the code, I added some patches here and there to make it work. Hence, the code is very likely not the most efficient way (I know it isn't) but it works fast enough and is flexible enough for me.

    I hadn't worked with XML before and I quickly got tired of trying to parse XML the way I assume it was intended. Instead I am simply looping through all the nodes and dumping the resulting columns of data into an Excel sheet and take if from there since I know I can get the data to dance pretty well once I have it in Excel.

    If you decide to have a look at it, I suggest that you first read the sheet called "Instructions" (go figure) since it should tell you most of what you need to know to use the workbook. Actually, it tells you a whole lot more than you need to know but I included the extra info in case some of you felt like actually creating your own customized formatting for how the odds are displayed in the workbook.

    Definitely works in Excel 2010, works in my version of Excel 2007 and probably works in Excel 2003. If you encounter any problems in running the macros (after having checked the references I list in the Instructions sheet) please let me know. Also, if it seems to run ok but the data looks incorrect I would definitely like to know. I have tested it a fair bit myself but not on all sports or all leagues so there might be some league specific issues that would prevent the data from displaying properly.

    Pinny says in the xml feed instructions that one can either load the entire xml or only the odds that have changed since last time. The main xml feed is only updated once every 10 mins, which is why one might not see any updated odds if re-importing the odds more frequently than that.

    I added the feature of importing only the changed odds. In the sheet called "User input", select "Full" to import all odds or "Incremental" to only import the odds that were changed since last time. Keep in mind that if you choose "Incremental", odds that were not updated will not show at all so it will appear that some available odds are missing.
    Attached Files
    Points Awarded:

    JohnAnthony gave thom321 2 SBR Point(s) for this post.

    flsaders85 gave thom321 2 SBR Point(s) for this post.

    doesnotcompute gave thom321 2 Betpoint(s) for this post.

    Nomination(s):
    This post was nominated 2 times . To view the nominated thread please click here. People who nominated: AlwaysDrawing, and doesnotcompute

  2. #2
    AlwaysDrawing
    AlwaysDrawing's Avatar Become A Pro!
    Join Date: 11-19-09
    Posts: 657
    Betpoints: 279

    Looks really nice. I'll take a closer look tomorrow, but very solid work.

  3. #3
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    I realize there is an issue with Excel cell formatting due to Excel automatically recognizing date formats and keeping the format as date even if the data in the cell is no longer a date. So if you use a display format other than the "Default" one I created, and then switch back to "Default", some columns will now be set to date format although they don't have a date in them. It happens to one or more of the columns with spread odds. The data is still there and you can manually change the cell format e.g. to General, or Number or whatever makes sense to see the data in the column.

    I am contemplating adding a feature to assign a cell format to each column in the display format section so that the cell formatting will match the type of data in the column so that switching between display formats will not cause any formatting issues. If I do, I'll post a new version here but it won't happen today.
    Points Awarded:

    JohnAnthony gave thom321 2 SBR Point(s) for this post.


  4. #4
    TomG
    TomG's Avatar Become A Pro!
    Join Date: 10-29-07
    Posts: 500

    Just use PivotTables to slice and dice the XML data. Then you can use GETPIVOTDATA() formulas to dynamically pull data from the PivotTables.

  5. #5
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    TomG,
    Are you saying that would make the workbook process the data faster than what it does currently? Plus, given the potentially large amounts of data (depending on how many sports or leagues you decide to display) I generally prefer to not have live formulas in cells, just the values.

  6. #6
    TomG
    TomG's Avatar Become A Pro!
    Join Date: 10-29-07
    Posts: 500

    not sure i don't use macros and don't want to download a workbook with macros in it. but my approach works not sure if it's better.

    edit: better meaning faster

  7. #7
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    TomG,
    Glad you pointed out that your comment was made without having seen how my Excel workbook works since to me it implied that my approach didn't work well (which it appears to do) and that Pivot Tables are better.

    Not sure about earlier versions but I believe Excel 2010 opens up a workbook with macros disabled by default (or you can at least set it up that way), which would allow anyone to open a workbook and at least view the layout, the formulas and the code, without risking that anything malicious would happen to the computer.

  8. #8
    TomG
    TomG's Avatar Become A Pro!
    Join Date: 10-29-07
    Posts: 500

    i took a look at the spreadsheet. no offense, but it's kind of a mess. i guess it's pretty good at gathering the entire xml data set and throwing it into a bunch of tabs. but pivottables are much better for organization a small amount of data such as a single sport. i'll upload my version sometime for comparison purposes.

  9. #9
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    TomG,

    Each to his/her own I guess. What is structured to one might be a mess to another.

    As a user, in my workbook you only need to care about two sheets, one where you pick the sports, leagues you want to view and how you want the output to look. The second sheet you would care about is the one where the odds tables are displayed, based on the display format selected by the user. I believe that is explained in the sheet called "Instructions".

    Everything else is there for "full disclosure" to anyone who wants to dig into how it works. It is also there for me to easily make changes or updates. I could have hidden all the sheets that are not necessary for the user to know about, which certainly would have made the look cleaner and obviously less confusing.

  10. #10
    AlwaysDrawing
    AlwaysDrawing's Avatar Become A Pro!
    Join Date: 11-19-09
    Posts: 657
    Betpoints: 279

    I'd be interested in seeing how your spreadsheet works too TomG.

  11. #11
    Greenei
    Greenei's Avatar Become A Pro!
    Join Date: 05-10-14
    Posts: 2
    Betpoints: 30

    Sorry for pushing this thread but I have a problem: Importing the data only works once and then does nothing before I restart Excel. Is there some bug in the code that could be fixed? I'm using Excel 2010.

  12. #12
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    Greenei,

    Due to the seeming lack of interest here I wasn't aware that anyone (other than me) was still using this. I just tried to import odds for MLB and it works fine for me. Could you please explain what you are trying to do and what is happening/not happening compared to what you expected to happen?

  13. #13
    Greenei
    Greenei's Avatar Become A Pro!
    Join Date: 05-10-14
    Posts: 2
    Betpoints: 30

    Oh, hey

    What I do specifically is selecting E-sports and GSL and sports on false and leagues on true and import type full. Then I click on Import Pinnacle Odds and then Create Odds table. This way it works fine and the odds table shows exactly what I can see on the website. But if I wait an hour or so and the odds change and I repeat the procedure the Odds table stays the same. What I would expect is a table with updated odds.

    If I delete the Odds table and just create Odds table again the same stuff that I just deleted appears again. So I think the problem lies with the import of the odds.

    The only way to fix this is exiting Excel and restarting it again. I plan to make a history of odds so that I can understand the odds movement. That's why I need it to work without closing and opening it again.
    Last edited by Greenei; 05-18-14 at 07:17 AM.

  14. #14
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    I can't recreate the issue you are having but it could have something to do with the xml file being retrieved from cache. Either way, it can generally be fixed. I have attached the most recent version I am using. Try it out and if you still have the same issue, i.e. odds are not imported again when you click on the "Import odds" button, let me know. I might be able to tweak it to force a fresh import each time.

    Pinnacle XML feed macro 05-18-2014.zip

    Quote Originally Posted by Greenei View Post
    Oh, hey

    What I do specifically is selecting E-sports and GSL and sports on false and leagues on true and import type full. Then I click on Import Pinnacle Odds and then Create Odds table. This way it works fine and the odds table shows exactly what I can see on the website. But if I wait an hour or so and the odds change and I repeat the procedure the Odds table stays the same. What I would expect is a table with updated odds.

    If I delete the Odds table and just create Odds table again the same stuff that I just deleted appears again. So I think the problem lies with the import of the odds.

    The only way to fix this is exiting Excel and restarting it again. I plan to make a history of odds so that I can understand the odds movement. That's why I need it to work without closing and opening it again.

  15. #15
    antonyp22
    antonyp22's Avatar Become A Pro!
    Join Date: 01-12-14
    Posts: 78
    Betpoints: 2528

    Very useful tool, thanks thom321!

  16. #16
    antonyp22
    antonyp22's Avatar Become A Pro!
    Join Date: 01-12-14
    Posts: 78
    Betpoints: 2528

    Could something similar be done with oddschecker.com?

  17. #17
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    Getting odds from oddschecker.com would require a very different approach than what I use in my workbook. Pinnacle provides the xml feed with all their odds which makes it relatively easy to retrieve them.

    I don't know anything about oddschecker.com but I would think it would be very detrimental to their business to provide such a feed (other than for a significant fee) since they are getting their odds by aggregating odds from bookmaker feeds. A quick look at their website tells me they are deliberately making it hard to scrape their odds, which makes perfect sense, from their perspective.

    It is definitely beyond my programming abilities to write something that would efficiently retrieve odds from Oddschecker's website.

    Quote Originally Posted by antonyp22 View Post
    Could something similar be done with oddschecker.com?

  18. #18
    Waz
    Waz's Avatar Become A Pro!
    Join Date: 12-25-08
    Posts: 262
    Betpoints: 1894

    Nice work

  19. #19
    lamichaeljames
    lamichaeljames's Avatar Become A Pro!
    Join Date: 06-02-14
    Posts: 40
    Betpoints: 109

    good work

    Good sheet

  20. #20
    shawn226
    shawn226's Avatar Become A Pro!
    Join Date: 06-25-11
    Posts: 18
    Betpoints: 24

    Pinnacle XML spreadsheet

    Hi Thom, I was googling around looking for a way to import Pinnacle's live XML feed into Excel when I found your post on SBR. Thom, does your spreadsheet create live charts of the Pinnacle odds also? I'm interested in viewing how the lines move and trend for MLB in order to try and jump in at a good price.

    Thanks!
    Shawn

  21. #21

  22. #22
    shawn226
    shawn226's Avatar Become A Pro!
    Join Date: 06-25-11
    Posts: 18
    Betpoints: 24

    fan-TASTIC Leetreaper, JUST what I was searching for... thanks SO much!!!!!

    Cheers
    Shawn

  23. #23
    sploofdogg
    sploofdogg's Avatar Become A Pro!
    Join Date: 01-20-13
    Posts: 335
    Betpoints: 1217

    Appreciate the link as well Lee.

    Question - So for example SDG/SEA, strictly ML, its showing RED for SEA and GREEN for SDG. Does this mean that $ at that moment is coming in on SEA and they are moving the line in that direction? Or opposite?

    Thanks,

    Sploofd

  24. #24
    Martinr
    Martinr's Avatar Become A Pro!
    Join Date: 07-08-13
    Posts: 529
    Betpoints: 21

    Green = money coming in/odds coming down.
    Red = odds drifting out.
    Click on odds (green or red) to see current trend.

  25. #25
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    Thanks leetreaper for the link.

    To address Shawn's question as it relates the workbook, only the most recent odds are shown. If someone wanted to program it to run every x minutes and store the imported odds somewhere, it is certainly possible. I have not interest in doing so since it would also have to include storing the data in a database of some sort as Excel wasn't made to store large amounts of data efficiently. If I really wanted to import/display odds history in Excel I would scrape the data from sites that have already done the job of storing the historical odds. It would be much more efficient that rewriting my workbook to store the price history.

  26. #26
    akphidelt
    akphidelt's Avatar Become A Pro!
    Join Date: 07-24-11
    Posts: 1,228
    Betpoints: 640

    Quote Originally Posted by thom321 View Post
    Thanks leetreaper for the link.

    To address Shawn's question as it relates the workbook, only the most recent odds are shown. If someone wanted to program it to run every x minutes and store the imported odds somewhere, it is certainly possible. I have not interest in doing so since it would also have to include storing the data in a database of some sort as Excel wasn't made to store large amounts of data efficiently. If I really wanted to import/display odds history in Excel I would scrape the data from sites that have already done the job of storing the historical odds. It would be much more efficient that rewriting my workbook to store the price history.
    Would there be a benefit to having this data? I can write a cron job to store the data in a database at any given increment which anyone can access.

  27. #27
    shawn226
    shawn226's Avatar Become A Pro!
    Join Date: 06-25-11
    Posts: 18
    Betpoints: 24

    Thanks Thom, but no need for this now... the Statsfair.com site that Leetreaper pointed out was exactly what I was searching for!

    Cheers
    Shawn

  28. #28
    akphidelt
    akphidelt's Avatar Become A Pro!
    Join Date: 07-24-11
    Posts: 1,228
    Betpoints: 640

    Sweeeeeeeet!! Thanks for the tip on the Pinnacle XML.

    I was able to write a cron job that picks up Pinnacle lines every minute and stores them in a database. Anyone have an idea on how to make this information useful? I can display it in any way possible if people have a plan.

    Edit: I changed it to 5 minutes. That's like 28k entries a day at a minute. I didn't realize Pinny put their lines up for future games so quickly.
    Last edited by akphidelt; 06-19-14 at 06:51 PM.

  29. #29
    akphidelt
    akphidelt's Avatar Become A Pro!
    Join Date: 07-24-11
    Posts: 1,228
    Betpoints: 640

    Alright guys, got home a little while ago and messed around. Got the Pinny results history in increments of 5 minutes. To get the results go to this page...

    http://akptracker.com/index.php?c=showodds

    This will be automatically updated to only include live odds and I have the Pinny cron job going every 5 minutes. If you click the "History" button for the game it will bring you to a table with all the odds I have recorded. It's actually pretty cool. Here's an example of the Red Sox/A's game tomorrow.

    http://akptracker.com/index.php?c=sh...meID=382235392

    Red Sox started at +156 now down to +147. Some cool stats there. I literally am just throwing this stuff together as I do not have much time. I understand the navigation is screwy and you have to push the back button. I'll get on all that later if this is something that people actually can use.

    Let me know if you have any suggestions?

  30. #30
    lamichaeljames
    lamichaeljames's Avatar Become A Pro!
    Join Date: 06-02-14
    Posts: 40
    Betpoints: 109

    nice..

  31. #31
    a4u2fear
    TEASE IT
    a4u2fear's Avatar SBR PRO
    Join Date: 01-29-10
    Posts: 8,147
    Betpoints: 35459

    I will likely go thru this and try to clean it up for myself for just NHL/NFL.

  32. #32
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    Quote Originally Posted by a4u2fear View Post
    I will likely go thru this and try to clean it up for myself for just NHL/NFL.
    Assuming you are talking about the Excel workbook created by me, you can select to only display odds for a specific sport or league. You can also create a custom display format to only show the columns you would like to see and your own descriptive headers for each column. The workbook has a few examples of custom display formats e.g. for baseball. I hope those examples are enough to create a custom display format for any sport. If not, let me know.

  33. #33
    a4u2fear
    TEASE IT
    a4u2fear's Avatar SBR PRO
    Join Date: 01-29-10
    Posts: 8,147
    Betpoints: 35459

    I know thom, I'm fluent with excel and VBA and there is too much action on the sheets and it is a little all over for me.

    but thanks for the sample XML coding, I am not familiar with it

  34. #34
    HUY
    HUY's Avatar Become A Pro!
    Join Date: 04-29-09
    Posts: 253
    Betpoints: 3257

    If you guys want any features added to Pinnacle Watch (www.statsfair.com/pinnacle) then let me know and I will try to add them once I have time.

  35. #35
    pringles
    pringles's Avatar Become A Pro!
    Join Date: 11-26-12
    Posts: 41
    Betpoints: 186

    Hey thom! I appreciate what you are doing!
    Would it be possible for you to write a sample vba code to grab Pinnacles API using HTTP Basic Authentication?

123 Last
Top