Together Tip: Large Import Performance
Thursday, December 13th, 2007As mentioned previously, I am committed to making Together perform really well, including with large libraries, as this is a common complaint from some people about apps of this sort (or so I keep hearing). As I find out about individual performance problems, I’m fixing them. It’s quite a challenge, people make incredible demands of these sorts of applications.
As I see it, the top priority here is to have Together perform well in everyday operations and it does, although I’ll mention some more improvements coming in that area below.
However, some people have criticised Together’s performance in importing a large number of files (anything from 5,000 to 20,000), so I’ll try to explain the problem here and give some tips on how to do that quickly and successfully.
Importing Large Numbers of Files
Together imports files in the background. This is useful for a number of reasons, not least as a way for you to use the Shelf without blocking the app with a progress sheet, and to keep the Shelf compatible with Spaces (showing a progress sheet on a window automatically makes that window’s space active, which is a real pain when you’re working in another application).
This background importing is also good because it means you can continue to work in the application while files are imported. However, there’s a downside in that, for every file added to the library, Together has to sort the list of files.
If the Library group (which shows all the files in Together) is selected and it already contains thousands of items, the second or so that it takes to sort the items adds considerably to the time it takes to import 10,000 files. And the more files there are in the library, the longer it will take to sort them. As such, large imports will get progressively slower.
There’s an easy answer to this: select a group or folder that is not affected by the large import and therefore doesn’t have to sort the incoming files. Using this method, I’ve imported 10,000 items in about 30 minutes, which is how long it took to copy them in the Finder.
In general, this is not an issue. How often do you import thousands of items a time? It’s typically a one-off operation performed when first using the software and most people don’t do it at all. That some people’s impression of the software is formed through this is unrepresentative and unfortunate. I can understand why they would feel frustrated.
Performance vs. Responsiveness
There is also the matter of performance vs. responsiveness. I have some changes lined up for the next version of Together in this area.
To give an example, somebody wrote to me yesterday to say that they got the spinning beachball when switching between some groups, but not others. It turns out the the first item in the slow group was an 8 megabyte PDF file (equates to about 200 pages). So, once the group was selected, Together would immediately start loading that PDF. Try opening an 8MB PDF in Preview and see how long it takes before it’s ready for you to start working with it.
In Together, most previews are loaded in the background and then displayed. This makes the app appear more responsive (i.e. you could scroll through many items or switch groups without a lag), but you don’t see the preview any quicker. Together also caches previews of some files so it’s quicker to revisit them.
I try to avoid these measures when possible because the background loading actually makes the app slower for some kinds of files and the caching increases the memory footprint (if you use too much memory, that will also slow the app down), but I will be taking this approach for large PDF files in the next version.