Geek asks and answers

Does RCHAR include READ_BYTES (proc / <pid> / io)?

I read proc/<pid>/ioto measure the IO activity of SQL queries, where <pid>is the PID of the database server. I read the values ​​before and after each request to calculate the difference, and get the number of bytes that the request received to read and / or write.

As far as I know, the field READ_BYTEScounts the actual disk-IO, but RCHARcontains more, for example, reads that can be satisfied by the Linux page cache (see Understanding the counters in / proc / [pid] / io for more details). This leads to the assumption that it RCHARshould have a value equal to or greater than READ_BYTES, but my results contradict this assumption.

I could imagine some minor overhead per block or page for the results I get for InfoBight ICE (MB values):

        Query        RCHAR   READ_BYTES
tpch_q01.sql|    34.44180|    34.89453|
tpch_q02.sql|     2.89191|     3.64453|
tpch_q03.sql|    32.58994|    33.19531|
tpch_q04.sql|    17.78325|    18.27344|

But I don't completely understand the IO counters for MonetDB (MB values):

        Query        RCHAR   READ_BYTES
tpch_q01.sql|     0.07501|   220.58203|
tpch_q02.sql|     1.37840|    18.16016|
tpch_q03.sql|     0.08272|   162.38281|
tpch_q04.sql|     0.06604|    83.25391|

Am I really mistaken in the assumption that RCHARincludes READ_BYTES? Is there a way to trick the kernel counters that MonetDB can use? What's going on here?

I can add that I clear the page cache and restart the database server before each request. I'm on Ubuntu 11.10, working with the 3.0.0-15-generic kernel.

+5
source share
2 answers

I can only think of two things:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/proc.txt;hb=HEAD#l1305

1

1446 read_bytes
1447 ----------
1448
1449 I/O counter: bytes read
1450 Attempt to count the number of bytes which this process really did cause to
1451 be fetched from the storage layer.

" ", readahead, .

2:

1411 rchar
1412 -----
1413
1414 I/O counter: chars read
1415 The number of bytes which this task has caused to be read from storage. This
1416 is simply the sum of bytes which this process passed to read() and pread().
1417 It includes things like tty IO and it is unaffected by whether or not actual
1418 physical disk IO was required (the read might have been satisfied from
1419 pagecache)

, " ". , , MonetDB, , .

, mmap .

+3

Linux: /include/linux/task_io_accounting.h

struct task_io_accounting {
#ifdef CONFIG_TASK_XACCT
  /* bytes read */
  u64 rchar;
  /*  bytes written */
  u64 wchar;
  /* # of read syscalls */
  u64 syscr;
  /* # of write syscalls */
  u64 syscw;
#endif /* CONFIG_TASK_XACCT */

#ifdef CONFIG_TASK_IO_ACCOUNTING
  /*
   * The number of bytes which this task has caused to be read from
   * storage.
   */
  u64 read_bytes;

  /*
   * The number of bytes which this task has caused, or shall cause to be
   * written to disk.
   */
  u64 write_bytes;

  /*
   * A task can cause "negative" IO too.  If this task truncates some
   * dirty pagecache, some IO which another task has been accounted for
   * (in its write_bytes) will not be happening.  We _could_ just
   * subtract that from the truncating task write_bytes, but there is
   * information loss in doing that.
   */
  u64 cancelled_write_bytes;
#endif /* CONFIG_TASK_IO_ACCOUNTING */
};
0

More articles:


All Articles