Forcing sequential processing in Haskell's Data.Binary.Get - performance

Question

After trying to import the basic Java runtime library rt.jar with language-java-classfile, I've discovered that it uses huge amounts of memory.
I've reduced the program demonstrating the problem to 100 lines and uploaded it to hpaste. Without forcing the evaluation of stream in line #94, I have no chance of ever running it because it eats up all my memory. Forcing stream before passing it to getClass finishes, but still uses up huge amounts of memory:
34,302,587,664 bytes allocated in the heap
32,583,990,728 bytes copied during GC
139,810,024 bytes maximum residency (398 sample(s))
29,142,240 bytes maximum slop
281 MB total memory in use (4 MB lost due to fragmentation)
Generation 0: 64992 collections, 0 parallel, 38.07s, 37.94s elapsed
Generation 1: 398 collections, 0 parallel, 25.87s, 27.78s elapsed
INIT time 0.01s ( 0.00s elapsed)
MUT time 37.22s ( 36.85s elapsed)
GC time 63.94s ( 65.72s elapsed)
RP time 0.00s ( 0.00s elapsed)
PROF time 13.00s ( 13.18s elapsed)
EXIT time 0.00s ( 0.00s elapsed)
Total time 114.17s (115.76s elapsed)
%GC time 56.0% (56.8% elapsed)
Alloc rate 921,369,531 bytes per MUT second
Productivity 32.6% of total user, 32.2% of total elapsed
I thought the problem was the ConstTables staying around, so I tried forcing cls in line #94 as well. But this only makes the memory consumption and the runtime worse:
34,300,700,520 bytes allocated in the heap
23,579,794,624 bytes copied during GC
487,798,904 bytes maximum residency (423 sample(s))
36,312,104 bytes maximum slop
554 MB total memory in use (10 MB lost due to fragmentation)
Generation 0: 64983 collections, 0 parallel, 71.19s, 71.48s elapsed
Generation 1: 423 collections, 0 parallel, 344.74s, 353.01s elapsed
INIT time 0.01s ( 0.00s elapsed)
MUT time 40.60s ( 42.38s elapsed)
GC time 415.93s (424.49s elapsed)
RP time 0.00s ( 0.00s elapsed)
PROF time 56.53s ( 57.71s elapsed)
EXIT time 0.00s ( 0.00s elapsed)
Total time 513.07s (524.58s elapsed)
%GC time 81.1% (80.9% elapsed)
Alloc rate 844,636,801 bytes per MUT second
Productivity 7.9% of total user, 7.7% of total elapsed
So my question is basically, how do I force sequential processing of the files involved, so that after each one is processed, only the string result (cls) remains in memory?

Answer 1

Edit 2: I just realized your code does this:
stream <- BL.pack <$> fileContents [] classfile
Don't do that. The pack functions are notoriously slow. You'll need to find a solution that doesn't involve using pack to create a ByteString.
I'm leaving the rest of my answer because I still think it applies, but this is almost certainly the biggest problem.
Unfortunately I can't test this because I don't recognize all your imports.
If you only want the result cls to remain in memory, why don't you force it instead of forcing stream? Change line 94 to
cls `seq` return cls
It may be necessary to use deepseq instead of just seq, although I have a suspicion that plain seq will be sufficient here.
However I think there's a better solution, and that's to use mapM_ instead of mapM. I think it's usually better style (and nearly always better performance) to create a function that does what it's supposed to with each result rather than returning a list. Here, you can change your main function to:
main = do
withArchive [CheckConsFlag] jarPath $ do
classfiles <- filter isClassfile <$> fileNames []
forM_ classfiles $ \classfile -> do
stream <- BL.pack <$> fileContents [] classfile
let cls = runGet getClass stream
lift $ print cls
Now the print is lifted into the function passed to forM_ for each classfile. The value cls is used internally and never returned, so it's both fully evaluated and quickly GC'd on each iteration of forM_.
Making use of this style in a larger application may require some refactoring or even redesign, but the results may be worth it.
Edit: If you're going to the trouble to redesign your code, you could use iteratees and avoid this problem entirely.

Answer 2

Your idea to force evaluation of cls in line 94 was right. But I guess you're approach to do so wasn't successfull. See this paste for my version, which runs in ca. 40MB instead of 220MB.
The key is to force reduction to normal form of cls, which is done by rnf cls. And this has to happen before the call to return. Therefore:
rnf cls `seq` return cls
Alternatively, you could use Control.Exception.evaluate:
evaluate $ rnf cls
return cls

Answer 3

Thanks for the suggestions.
I think for my concrete problem, the solution will be to process .jar files in small chunks -- fortunately, inner classes are always in the same dir in the .jar file as their outer class, so there is no need to process all 50 megs in one run.
The only thing I couldn't quite understand is if it is possible to use libzip via enumerators, or would that need a new libzip implementation?