Geek asks and answers

Can I read n files lazily as a single I / O operation in Haskell?

How can I read multiple files as one ByteString lazily with read-only memory?

readFiles :: [FilePath] -> IO ByteString

Currently I have the following implementation, but from what I saw from profiling, and also because of my understanding, I end up n-1files in memory.

readFiles = foldl1 joinIOStrings . map ByteString.readFile
    where joinIOStrings ml mr = do
                                l <- ml
                                r <- mr
                                return $ l `ByteString.append` r

I understand that the drawback here is that I apply the IO actions and then replace them, so I think I need a way to replace foldl1 joinIOStringswithout applying them.

+5
source share
2 answers

How can I read multiple files as one ByteString lazily with read-only memory?

, Data.ByteString.Lazy. ByteString O(sum of filesizes) .

, (D.B.L.readFile ) ,

import qualified Data.ByteString.Lazy as L

readFiles :: [FilePath] -> IO L.ByteString
readFiles = fmap L.concat . mapM L.readFile

mapM L.readFile , , .

, , , , - . mapM,

import System.IO.Unsafe (unsafeInterleaveIO)

mapM_lazy :: [IO a] -> IO [a]
mapM_lazy [] = return []
mapM_lazy (x:xs) = do
              r <- x
              rs <- unsafeInterleaveIO (mapM_lazy xs)
              return (r:rs)

, , . , , .

iteratee, enumerator, conduit , . , , .

+7

, ( Data.ByteString.Lazy). , , - concat :: [ByteString] -> ByteString:

import Control.Monad
import Data.ByteString.Lazy (ByteString)
import qualified Data.ByteString.Lazy as ByteString

readFiles :: [FilePath] -> IO ByteString
readFiles = fmap ByteString.concat . mapM ByteString.readFile

(: , , )

+1

More articles:


All Articles