Using Excel to import and filter all Pinnacle lines by sport/league

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • thom321
    SBR High Roller
    • 06-17-11
    • 112

    #1
    Using Excel to import and filter all Pinnacle lines by sport/league
    uva3021's work on grabbing Pinnacle's XML feed with Excel gave me an idea to take this a step further by building a workbook that allows the user to create a table with odds for any sport/league that Pinnacle offers odds for and displaying it in a customized format e.g. depending on the sport or event. Since Pinnacle explicitly allows anyone to access their XML feed once every 60 seconds, I figured too much harm can't be done by sharing this. Also, if you were to change the settings to import it more often, I am pretty sure your IP would eventually get blocked by Pinny. Obviously there are ways around that too but my point is that as long as everything in the workbook is left the way I set it up, no rules (explicit or implicit) are likely to be broken.

    My goal was to get a working solution, not the prettiest, and in its current version it works for me at least and maybe it can be useful for someone else.

    I wouldn't advice using the code I created as a proper way of writing VBA since my focus was on getting something working quickly. When that failed and the task got infinitely more complicated than I envisioned initially, instead of re-writing much of the code, I added some patches here and there to make it work. Hence, the code is very likely not the most efficient way (I know it isn't) but it works fast enough and is flexible enough for me.

    I hadn't worked with XML before and I quickly got tired of trying to parse XML the way I assume it was intended. Instead I am simply looping through all the nodes and dumping the resulting columns of data into an Excel sheet and take if from there since I know I can get the data to dance pretty well once I have it in Excel.

    If you decide to have a look at it, I suggest that you first read the sheet called "Instructions" (go figure) since it should tell you most of what you need to know to use the workbook. Actually, it tells you a whole lot more than you need to know but I included the extra info in case some of you felt like actually creating your own customized formatting for how the odds are displayed in the workbook.

    Definitely works in Excel 2010, works in my version of Excel 2007 and probably works in Excel 2003. If you encounter any problems in running the macros (after having checked the references I list in the Instructions sheet) please let me know. Also, if it seems to run ok but the data looks incorrect I would definitely like to know. I have tested it a fair bit myself but not on all sports or all leagues so there might be some league specific issues that would prevent the data from displaying properly.

    Pinny says in the xml feed instructions that one can either load the entire xml or only the odds that have changed since last time. The main xml feed is only updated once every 10 mins, which is why one might not see any updated odds if re-importing the odds more frequently than that.

    I added the feature of importing only the changed odds. In the sheet called "User input", select "Full" to import all odds or "Incremental" to only import the odds that were changed since last time. Keep in mind that if you choose "Incremental", odds that were not updated will not show at all so it will appear that some available odds are missing.
    Attached Files
  • AlwaysDrawing
    SBR Wise Guy
    • 11-20-09
    • 657

    #2
    Looks really nice. I'll take a closer look tomorrow, but very solid work.
    Comment
    • thom321
      SBR High Roller
      • 06-17-11
      • 112

      #3
      I realize there is an issue with Excel cell formatting due to Excel automatically recognizing date formats and keeping the format as date even if the data in the cell is no longer a date. So if you use a display format other than the "Default" one I created, and then switch back to "Default", some columns will now be set to date format although they don't have a date in them. It happens to one or more of the columns with spread odds. The data is still there and you can manually change the cell format e.g. to General, or Number or whatever makes sense to see the data in the column.

      I am contemplating adding a feature to assign a cell format to each column in the display format section so that the cell formatting will match the type of data in the column so that switching between display formats will not cause any formatting issues. If I do, I'll post a new version here but it won't happen today.
      Comment
      • TomG
        SBR Wise Guy
        • 10-29-07
        • 500

        #4
        Just use PivotTables to slice and dice the XML data. Then you can use GETPIVOTDATA() formulas to dynamically pull data from the PivotTables.
        Comment
        • thom321
          SBR High Roller
          • 06-17-11
          • 112

          #5
          TomG,
          Are you saying that would make the workbook process the data faster than what it does currently? Plus, given the potentially large amounts of data (depending on how many sports or leagues you decide to display) I generally prefer to not have live formulas in cells, just the values.
          Comment
          • TomG
            SBR Wise Guy
            • 10-29-07
            • 500

            #6
            not sure i don't use macros and don't want to download a workbook with macros in it. but my approach works not sure if it's better.

            edit: better meaning faster
            Comment
            • thom321
              SBR High Roller
              • 06-17-11
              • 112

              #7
              TomG,
              Glad you pointed out that your comment was made without having seen how my Excel workbook works since to me it implied that my approach didn't work well (which it appears to do) and that Pivot Tables are better.

              Not sure about earlier versions but I believe Excel 2010 opens up a workbook with macros disabled by default (or you can at least set it up that way), which would allow anyone to open a workbook and at least view the layout, the formulas and the code, without risking that anything malicious would happen to the computer.
              Comment
              • TomG
                SBR Wise Guy
                • 10-29-07
                • 500

                #8
                i took a look at the spreadsheet. no offense, but it's kind of a mess. i guess it's pretty good at gathering the entire xml data set and throwing it into a bunch of tabs. but pivottables are much better for organization a small amount of data such as a single sport. i'll upload my version sometime for comparison purposes.
                Comment
                • thom321
                  SBR High Roller
                  • 06-17-11
                  • 112

                  #9
                  TomG,

                  Each to his/her own I guess. What is structured to one might be a mess to another.

                  As a user, in my workbook you only need to care about two sheets, one where you pick the sports, leagues you want to view and how you want the output to look. The second sheet you would care about is the one where the odds tables are displayed, based on the display format selected by the user. I believe that is explained in the sheet called "Instructions".

                  Everything else is there for "full disclosure" to anyone who wants to dig into how it works. It is also there for me to easily make changes or updates. I could have hidden all the sheets that are not necessary for the user to know about, which certainly would have made the look cleaner and obviously less confusing.
                  Comment
                  • AlwaysDrawing
                    SBR Wise Guy
                    • 11-20-09
                    • 657

                    #10
                    I'd be interested in seeing how your spreadsheet works too TomG.
                    Comment
                    • Greenei
                      Restricted User
                      • 05-10-14
                      • 2

                      #11
                      Sorry for pushing this thread but I have a problem: Importing the data only works once and then does nothing before I restart Excel. Is there some bug in the code that could be fixed? I'm using Excel 2010.
                      Comment
                      • thom321
                        SBR High Roller
                        • 06-17-11
                        • 112

                        #12
                        Greenei,

                        Due to the seeming lack of interest here I wasn't aware that anyone (other than me) was still using this. I just tried to import odds for MLB and it works fine for me. Could you please explain what you are trying to do and what is happening/not happening compared to what you expected to happen?
                        Comment
                        • Greenei
                          Restricted User
                          • 05-10-14
                          • 2

                          #13
                          Oh, hey

                          What I do specifically is selecting E-sports and GSL and sports on false and leagues on true and import type full. Then I click on Import Pinnacle Odds and then Create Odds table. This way it works fine and the odds table shows exactly what I can see on the website. But if I wait an hour or so and the odds change and I repeat the procedure the Odds table stays the same. What I would expect is a table with updated odds.

                          If I delete the Odds table and just create Odds table again the same stuff that I just deleted appears again. So I think the problem lies with the import of the odds.

                          The only way to fix this is exiting Excel and restarting it again. I plan to make a history of odds so that I can understand the odds movement. That's why I need it to work without closing and opening it again.
                          Last edited by Greenei; 05-18-14, 07:17 AM.
                          Comment
                          • thom321
                            SBR High Roller
                            • 06-17-11
                            • 112

                            #14
                            I can't recreate the issue you are having but it could have something to do with the xml file being retrieved from cache. Either way, it can generally be fixed. I have attached the most recent version I am using. Try it out and if you still have the same issue, i.e. odds are not imported again when you click on the "Import odds" button, let me know. I might be able to tweak it to force a fresh import each time.

                            Pinnacle XML feed macro 05-18-2014.zip

                            Originally posted by Greenei
                            Oh, hey

                            What I do specifically is selecting E-sports and GSL and sports on false and leagues on true and import type full. Then I click on Import Pinnacle Odds and then Create Odds table. This way it works fine and the odds table shows exactly what I can see on the website. But if I wait an hour or so and the odds change and I repeat the procedure the Odds table stays the same. What I would expect is a table with updated odds.

                            If I delete the Odds table and just create Odds table again the same stuff that I just deleted appears again. So I think the problem lies with the import of the odds.

                            The only way to fix this is exiting Excel and restarting it again. I plan to make a history of odds so that I can understand the odds movement. That's why I need it to work without closing and opening it again.
                            Comment
                            • antonyp22
                              SBR Hustler
                              • 01-12-14
                              • 78

                              #15
                              Very useful tool, thanks thom321!
                              Comment
                              • antonyp22
                                SBR Hustler
                                • 01-12-14
                                • 78

                                #16
                                Could something similar be done with oddschecker.com?
                                Comment
                                • thom321
                                  SBR High Roller
                                  • 06-17-11
                                  • 112

                                  #17
                                  Getting odds from oddschecker.com would require a very different approach than what I use in my workbook. Pinnacle provides the xml feed with all their odds which makes it relatively easy to retrieve them.

                                  I don't know anything about oddschecker.com but I would think it would be very detrimental to their business to provide such a feed (other than for a significant fee) since they are getting their odds by aggregating odds from bookmaker feeds. A quick look at their website tells me they are deliberately making it hard to scrape their odds, which makes perfect sense, from their perspective.

                                  It is definitely beyond my programming abilities to write something that would efficiently retrieve odds from Oddschecker's website.

                                  Originally posted by antonyp22
                                  Could something similar be done with oddschecker.com?
                                  Comment
                                  • Waz
                                    SBR Sharp
                                    • 12-25-08
                                    • 262

                                    #18
                                    Nice work
                                    Comment
                                    • lamichaeljames
                                      SBR Rookie
                                      • 06-02-14
                                      • 40

                                      #19
                                      good work

                                      Good sheet
                                      Comment
                                      • shawn226
                                        SBR Rookie
                                        • 06-25-11
                                        • 18

                                        #20
                                        Pinnacle XML spreadsheet

                                        Hi Thom, I was googling around looking for a way to import Pinnacle's live XML feed into Excel when I found your post on SBR. Thom, does your spreadsheet create live charts of the Pinnacle odds also? I'm interested in viewing how the lines move and trend for MLB in order to try and jump in at a good price.

                                        Thanks!
                                        Shawn
                                        Comment
                                        • leetreaper
                                          BARRELED IN @ SBR!
                                          • 10-23-10
                                          • 34841

                                          #21
                                          enjoy
                                          Comment
                                          • shawn226
                                            SBR Rookie
                                            • 06-25-11
                                            • 18

                                            #22
                                            fan-TASTIC Leetreaper, JUST what I was searching for... thanks SO much!!!!!

                                            Cheers
                                            Shawn
                                            Comment
                                            • sploofdogg
                                              SBR Sharp
                                              • 01-20-13
                                              • 335

                                              #23
                                              Appreciate the link as well Lee.

                                              Question - So for example SDG/SEA, strictly ML, its showing RED for SEA and GREEN for SDG. Does this mean that $ at that moment is coming in on SEA and they are moving the line in that direction? Or opposite?

                                              Thanks,

                                              Sploofd
                                              Comment
                                              • Martinr
                                                SBR Wise Guy
                                                • 07-08-13
                                                • 529

                                                #24
                                                Green = money coming in/odds coming down.
                                                Red = odds drifting out.
                                                Click on odds (green or red) to see current trend.
                                                Comment
                                                • thom321
                                                  SBR High Roller
                                                  • 06-17-11
                                                  • 112

                                                  #25
                                                  Thanks leetreaper for the link.

                                                  To address Shawn's question as it relates the workbook, only the most recent odds are shown. If someone wanted to program it to run every x minutes and store the imported odds somewhere, it is certainly possible. I have not interest in doing so since it would also have to include storing the data in a database of some sort as Excel wasn't made to store large amounts of data efficiently. If I really wanted to import/display odds history in Excel I would scrape the data from sites that have already done the job of storing the historical odds. It would be much more efficient that rewriting my workbook to store the price history.
                                                  Comment
                                                  • akphidelt
                                                    SBR MVP
                                                    • 07-24-11
                                                    • 1228

                                                    #26
                                                    Originally posted by thom321
                                                    Thanks leetreaper for the link.

                                                    To address Shawn's question as it relates the workbook, only the most recent odds are shown. If someone wanted to program it to run every x minutes and store the imported odds somewhere, it is certainly possible. I have not interest in doing so since it would also have to include storing the data in a database of some sort as Excel wasn't made to store large amounts of data efficiently. If I really wanted to import/display odds history in Excel I would scrape the data from sites that have already done the job of storing the historical odds. It would be much more efficient that rewriting my workbook to store the price history.
                                                    Would there be a benefit to having this data? I can write a cron job to store the data in a database at any given increment which anyone can access.
                                                    Comment
                                                    • shawn226
                                                      SBR Rookie
                                                      • 06-25-11
                                                      • 18

                                                      #27
                                                      Thanks Thom, but no need for this now... the Statsfair.com site that Leetreaper pointed out was exactly what I was searching for!

                                                      Cheers
                                                      Shawn
                                                      Comment
                                                      • akphidelt
                                                        SBR MVP
                                                        • 07-24-11
                                                        • 1228

                                                        #28
                                                        Sweeeeeeeet!! Thanks for the tip on the Pinnacle XML.

                                                        I was able to write a cron job that picks up Pinnacle lines every minute and stores them in a database. Anyone have an idea on how to make this information useful? I can display it in any way possible if people have a plan.

                                                        Edit: I changed it to 5 minutes. That's like 28k entries a day at a minute. I didn't realize Pinny put their lines up for future games so quickly.
                                                        Last edited by akphidelt; 06-19-14, 06:51 PM.
                                                        Comment
                                                        • akphidelt
                                                          SBR MVP
                                                          • 07-24-11
                                                          • 1228

                                                          #29
                                                          Alright guys, got home a little while ago and messed around. Got the Pinny results history in increments of 5 minutes. To get the results go to this page...



                                                          This will be automatically updated to only include live odds and I have the Pinny cron job going every 5 minutes. If you click the "History" button for the game it will bring you to a table with all the odds I have recorded. It's actually pretty cool. Here's an example of the Red Sox/A's game tomorrow.



                                                          Red Sox started at +156 now down to +147. Some cool stats there. I literally am just throwing this stuff together as I do not have much time. I understand the navigation is screwy and you have to push the back button. I'll get on all that later if this is something that people actually can use.

                                                          Let me know if you have any suggestions?
                                                          Comment
                                                          • lamichaeljames
                                                            SBR Rookie
                                                            • 06-02-14
                                                            • 40

                                                            #30
                                                            nice..
                                                            Comment
                                                            • a4u2fear
                                                              SBR Hall of Famer
                                                              • 01-29-10
                                                              • 8147

                                                              #31
                                                              I will likely go thru this and try to clean it up for myself for just NHL/NFL.
                                                              Comment
                                                              • thom321
                                                                SBR High Roller
                                                                • 06-17-11
                                                                • 112

                                                                #32
                                                                Originally posted by a4u2fear
                                                                I will likely go thru this and try to clean it up for myself for just NHL/NFL.
                                                                Assuming you are talking about the Excel workbook created by me, you can select to only display odds for a specific sport or league. You can also create a custom display format to only show the columns you would like to see and your own descriptive headers for each column. The workbook has a few examples of custom display formats e.g. for baseball. I hope those examples are enough to create a custom display format for any sport. If not, let me know.
                                                                Comment
                                                                • a4u2fear
                                                                  SBR Hall of Famer
                                                                  • 01-29-10
                                                                  • 8147

                                                                  #33
                                                                  I know thom, I'm fluent with excel and VBA and there is too much action on the sheets and it is a little all over for me.

                                                                  but thanks for the sample XML coding, I am not familiar with it
                                                                  Comment
                                                                  • HUY
                                                                    SBR Sharp
                                                                    • 04-29-09
                                                                    • 253

                                                                    #34
                                                                    If you guys want any features added to Pinnacle Watch (www.statsfair.com/pinnacle) then let me know and I will try to add them once I have time.
                                                                    Comment
                                                                    • pringles
                                                                      SBR Rookie
                                                                      • 11-26-12
                                                                      • 41

                                                                      #35
                                                                      Hey thom! I appreciate what you are doing!
                                                                      Would it be possible for you to write a sample vba code to grab Pinnacles API using HTTP Basic Authentication?
                                                                      Comment
                                                                      SBR Contests
                                                                      Collapse
                                                                      Top-Rated US Sportsbooks
                                                                      Collapse
                                                                      Working...