Geek asks and answers

Requires file system metadata level for applications

I'm looking for a metadata layer that sits on top of files that can interpret pairs of key values ​​in file names for applications that work with thousands of files. Additional Information:

  • These are not necessarily media files with embedded metadata — hence key-value pairs.
  • Metadata goes beyond os-information (file sizes, etc.) - regardless of what the application puts in the key values.
  • It should be accessible through the command line as well as the python module so that my applications can talk to it.
  • ADDED: it should also be supported by the general os commands (cp, mv, tar, etc.) so that it is not lost if the file is copied or moved.

Examples of functionality that I would like to include:

  • list of files in the x directory for organization_id 3375
  • report the files in the y directory by converting load_time to year / month and specify the number of files and file size for each year / month.
  • get the oldest file in the z directory based on the download time key

Files with this simple metadata embedded in them might look like this:

  • bowling_state-ky_league-15_game-8_gametime-201209141830.tgz
  • bowling_state-ky_league-15_game-9_gametime-201209141930.tgz

This metadata is very accessible and closely related to the file. But - I would prefer to avoid using cut or wild cards for all operations.

I looked around and can only find media and os-metadata and I do not want to create something if it already exists.

+5
source share
3 answers

? .: http://en.wikipedia.org/wiki/Extended_file_attributes

, - . :

$ setfattr -n user.comment -v "this is a comment" testfile
$ getfattr testfile
# file: testfile
user.comment
$ getfattr -n user.comment testfile
# file: testfile
user.comment="this is a comment"

python, python xattr. .: http://pypi.python.org/pypi/xattr

, cp, mv tar, . . cp -a tar -xattr. , . (, , .) , . cp = "cp -a".

+3

, xattrs - , . , xattrs:

NTFS

Microsoft Windows xattrs , NTFS . ADS .

drive:\path\to\file:streamname

ADS - . , Python, , :

open(r"drive:\path\to\file:streamname", "wb")

. ( : .)

Microsoft streams.

ADS , .

SQLite

SQLite - , . .sqlite .

:

CREATE TABLE file (
    file_id INTEGER PRIMARY KEY AUTOINCREMENT,
    path TEXT
);

, , :

CREATE TABLE organization_id (
    file_id INTEGER PRIMARY KEY,
    value INTEGER,
    FOREIGN KEY(file_id) REFERENCES file(file_id)
);

:

SELECT path FROM file NATURAL JOIN organization_id
WHERE value == 3375 AND path LIKE '/x/%';

, , :

CREATE TABLE metadata (
    file_id INTEGER,
    key TEXT,
    value TEXT,
    PRIMARY KEY(file_id, key),
    FOREIGN KEY(file_id) REFERENCES file(file_id)
);

Query:

SELECT path FROM file NATURAL JOIN metadata
WHERE key == 'organization_id' AND value == 3375 AND path LIKE '/x/%';

, , . , (, active file), : set active = FALSE, , fsync, active = TRUE active = FALSE).

+2

Xattr (Ext4 4kb), xattr "". linux. xattr.

iDB.py , xattr xattr.

0
source

More articles:


All Articles