1. #1
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990

    What to do when Excel is too slow?

    What do people do when Excel isn't getting it done? Keep getting "resources not available" errors... mysql or something like that seems to lack the computational power of Excel with all the sumifs and similar valuable functions. Any input appreciated... thanks!

  2. #2
    Sawyer
    Sawyer's Avatar Become A Pro!
    Join Date: 06-01-09
    Posts: 7,592
    Betpoints: 6650

    What to do when Excel is too slow?

    I wait. Hehe.

    Use i7 computer.

  3. #3
    mathdotcom
    mathdotcom's Avatar Become A Pro!
    Join Date: 03-24-08
    Posts: 11,689
    Betpoints: 1943

    Invest financially and mentally in a command based program. Which one I'd suggest depends on your needs. Any will have those Excel functions and will handle larger datasets given your computer's specs.

  4. #4
    Emily_Haines
    Emily_Haines's Avatar Become A Pro!
    Join Date: 04-14-09
    Posts: 15,888
    Betpoints: 15313

    Quote Originally Posted by roasthawg View Post
    What do people do when Excel isn't getting it done? Keep getting "resources not available" errors... mysql or something like that seems to lack the computational power of Excel with all the sumifs and similar valuable functions. Any input appreciated... thanks!
    If you have data utilizing lots of formulas this will drastically slow everything down.

  5. #5
    mathdotcom
    mathdotcom's Avatar Become A Pro!
    Join Date: 03-24-08
    Posts: 11,689
    Betpoints: 1943

    Excel is even exceptionally slow as a basic spreadsheet program. Takes forever to open.

    In terms of analyzing data of any substantial size, I feel like my computer is suffering a horrible horrible death.

  6. #6
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    Depending on how your data is structured and how large your data set is, some simple VBA macros could make a big difference. If you have a lot of formulas that repeat throughout the entire set of data, you could have whatever formulas you are using in the top row, then use VBA to apply the formula in the top row to the entire data set but instead of having the formulas "live" on each row (which could bring Excel to a halt especially with functions that use the entire data set like sumifs, countifs) the formulas will be replaced with values in every row except the top row where you keep the formula the way you want it.
    Last edited by thom321; 07-05-12 at 12:09 AM.

  7. #7
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Learn a programming language and/or learn/interface with a statistical language like R, Matlab, etc.

    Excel is pretty much worthless beyond fundamental exercises.

  8. #8
    tukkk
    ★★★★★
    tukkk's Avatar Become A Pro!
    Join Date: 10-04-10
    Posts: 391

    as thom said, vba macros could make a big difference

    and if it is possible you should break your dataset into smaller pieces

  9. #9
    kilmerAsHolliday
    kilmerAsHolliday's Avatar Become A Pro!
    Join Date: 11-17-09
    Posts: 45
    Betpoints: 192

    I went xl->xl/vba->visualStudio->xl/vba->php/mysql
    I recommend skipping the middle bit if at all possible.

  10. #10
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Quote Originally Posted by roasthawg View Post
    What do people do when Excel isn't getting it done? Keep getting "resources not available" errors... mysql or something like that seems to lack the computational power of Excel with all the sumifs and similar valuable functions. Any input appreciated... thanks!
    By the way, there are ways to sum in SQL (and quite a bit faster than Excel)...

    SELECT SUM(CASE WHEN Column2 = 'Some Value' THEN Column1 ELSE 0 END) as sum FROM table_name

    That will select and sum all records in Column1 if Column2 meets a criterion of "Some Value".

  11. #11
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Quote Originally Posted by kilmerAsHolliday View Post
    I went xl->xl/vba->visualStudio->xl/vba->php/mysql
    I recommend skipping the middle bit if at all possible.
    Unless multithreading (and utilizing multi-core processors to capacity) isn't important to someone, I wouldn't skip or disregard the VS languages at all.

    It's one of the many reasons that Excel lacks any speed.

  12. #12
    kilmerAsHolliday
    kilmerAsHolliday's Avatar Become A Pro!
    Join Date: 11-17-09
    Posts: 45
    Betpoints: 192

    Quote Originally Posted by MonkeyF0cker View Post
    Unless multithreading (and utilizing multi-core processors to capacity) isn't important to someone, I wouldn't skip or disregard the VS languages at all.

    It's one of the many reasons that Excel lacks any speed.
    pretty easy to curl ur own scripts. voila!

  13. #13
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Quote Originally Posted by kilmerAsHolliday View Post
    pretty easy to curl ur own scripts. voila!
    Yeah. cURL has come a long way, but it still has (and always will have) some serious limitations. A language with native multithreading makes many, many tasks much simpler and efficient.

  14. #14
    Pokerjoe
    Pokerjoe's Avatar Become A Pro!
    Join Date: 04-17-09
    Posts: 704
    Betpoints: 307

    Open your worksheet.

    Press CTRL END.

    If you end up at the bottom of the whole sheet, and especially if this is the case on numerous sheets, that's the problem. I've faced it before a few times, I think generally from Copying and Pasting entire columns. When the entire gazillion available cells in a sheet have to be checked every time you do anything, the system bogs down.

    Copy the cells you actually need to work into a clean sheet, delete the old sheet, and you should be fine. For example, if the cells you're actively using are, say, ending at JY106, but CTRL END brings you to whatever, row 100,000+, etc, then just completely copy from A1 to JY106 into a new sheet, then delete the old sheet.

    If your data is filling up entire sheets, then yes, you have to change to something other than Excel.
    Nomination(s):
    This post was nominated 1 time . To view the nominated thread please click here. People who nominated: TomG

  15. #15
    Lookingtostart
    Lookingtostart's Avatar Become A Pro!
    Join Date: 04-25-11
    Posts: 1,584
    Betpoints: 4542

    Have you tried Libre Office? You can change settings relating to RAM usage as well as being more efficient IMO.

  16. #16
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990

    Quote Originally Posted by thom321 View Post
    Depending on how your data is structured and how large your data set is, some simple VBA macros could make a big difference. If you have a lot of formulas that repeat throughout the entire set of data, you could have whatever formulas you are using in the top row, then use VBA to apply the formula in the top row to the entire data set but instead of having the formulas "live" on each row (which could bring Excel to a halt especially with functions that use the entire data set like sumifs, countifs) the formulas will be replaced with values in every row except the top row where you keep the formula the way you want it.
    So basically automate paste specials?

  17. #17
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990

    Quote Originally Posted by mathdotcom View Post
    Invest financially and mentally in a command based program. Which one I'd suggest depends on your needs. Any will have those Excel functions and will handle larger datasets given your computer's specs.
    Can you give me examples of a program like this?

  18. #18
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990

    Quote Originally Posted by MonkeyF0cker View Post
    Learn a programming language and/or learn/interface with a statistical language like R, Matlab, etc.

    Excel is pretty much worthless beyond fundamental exercises.
    Already have a working knowledge of R and PHP... neither have the computational convenience of excel and it's formulas such as sumifs to my knowledge.

  19. #19
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990

    Quote Originally Posted by MonkeyF0cker View Post
    By the way, there are ways to sum in SQL (and quite a bit faster than Excel)...

    SELECT SUM(CASE WHEN Column2 = 'Some Value' THEN Column1 ELSE 0 END) as sum FROM table_name

    That will select and sum all records in Column1 if Column2 meets a criterion of "Some Value".
    This might be the way to go for me... I'll play with this and see how it works for me. Thanks!

  20. #20
    roasthawg
    roasthawg's Avatar Become A Pro!
    Join Date: 11-09-07
    Posts: 2,990

    Quote Originally Posted by kilmerAsHolliday View Post
    I went xl->xl/vba->visualStudio->xl/vba->php/mysql
    I recommend skipping the middle bit if at all possible.
    Good news for me as I know php and a bit of mysql. Easy for me to put information into mysql via php... much tougher to analyze and manipulate it.

  21. #21
    thom321
    thom321's Avatar Become A Pro!
    Join Date: 06-17-11
    Posts: 112
    Betpoints: 4983

    "So basically automate paste specials?"

    Functionally yes, but I wouldn't actually use paste special in the code.
    The code doing the "copy" and "paste" could look something like:

    'Copy range with formula. Paste in destination range
    rngCopyRange.Copy rngDestRange
    'Recalculate sheet to update formula values assuming Excel calculation was set to manual at the beginning of the procedure
    Calculate
    'Suggest including DoEvents to avoid Excel getting stuck or hanging when looping through rows
    DoEvents
    'replace with values
    rngDestRange.Value = rngDestRange.Value

  22. #22
    Justin7
    Justin7's Avatar Become A Pro!
    Join Date: 07-31-06
    Posts: 8,577
    Betpoints: 1506

    Quote Originally Posted by MonkeyF0cker View Post
    Learn a programming language and/or learn/interface with a statistical language like R, Matlab, etc.

    Excel is pretty much worthless beyond fundamental exercises.
    One big advantage of Excel is its universality. It's obviously not as fast or flexible as other languages. But, it is much more portable. If you are working with other people, translating your work to (or starting a project in) Excel makes collaborating much easier.

  23. #23
    MonkeyF0cker
    Update your status
    MonkeyF0cker's Avatar Become A Pro!
    Join Date: 06-12-07
    Posts: 12,144
    Betpoints: 1127

    Quote Originally Posted by Justin7 View Post
    One big advantage of Excel is its universality. It's obviously not as fast or flexible as other languages. But, it is much more portable. If you are working with other people, translating your work to (or starting a project in) Excel makes collaborating much easier.
    I don't see the point if Excel can't handle whatever you're doing. If whoever you're working with can't "translate" your work easily, that's just another reason why collaborating is completely inefficient. It's not hard to zip up a database and source, encrypt it, and ship it off to someone if "translating" isn't an issue.

Top