Geek asks and answers

How do you adjust the concurrency / relative performance of a process in Erlang?

Let's say I need to read from a directory that has a lot of large XML files, and I have to parse this and send them to some service via the network, and then write the answer back to disk.

If it was Java or C ++ etc., I can do something like this (hopefully this makes sense):

(File read & xml parsing process) -> bounded-queue -> (sender process) -> service

service -> bounded-queue -> (process to parse result and write to disk)

And then I will assign any suitable number of threads to each process. Thus, I can limit the concurrency of each process to an optimal value, and a limited queue will guarantee that there is no memory shortage, etc.

What should I do if coding in Erlang? I assume that I could just implement the whole thread in a function, then iterate over the directory and start these “initial” processes as quickly as possible. This sounds suboptimal, because if parsing XML takes longer than reading files, etc., the application. may run out of memory because you have many XML documents, etc. at the same time, and you cannot maintain concurrency at the optimal level. For instance. if the “service” is most effective when the concurrency is 4, it would be very inefficient to hit it with a huge concurrency.

How will erlang programmers deal with this situation? That is, what is the erlang alternative for a fixed thread pool and a limited queue?

+3
3

, . , . , , . , , ect.

. :

pool(Max) -> 
    process_flag(trap_exit, true),
    pool(0, Max);
pool(Current, Max) ->
    receive
        {'EXIT', _, _} -> 
            pool(Current - 1, Max);
        { work, F, Pid} when Current < Max -> 
            Pid ! accepted,
            spawn_link(F),
            pool(Current + 1, Max);
        { work, _, Pid} -> 
            Pid ! rejected,
            pool(Current, Max);
    end.

, , . , , .

+2

Erlang, , XML- ( , ).

, , , . , (: memsup).

+3

.

, OTP gen_server .

  • gen_servers , , .

  • gen_server ( EXIT).

  • gen_server 1 , .

, , gen_server .

, , gen_servers , , , . , , - NFS, .

, concurrency.

+1

More articles:


All Articles