iRODS User Guide


iRODS – integrated Rule-Oriented Data-management System

What is iRODS

iRODS allows the storage and sharing of large data sets, with the ability to tag/add meta data. (iRODS Home)

iRODS is running on archive1.cc.dtu.dk and data replicated to archive2.cc.dtu.dk. Access to the servers is restricted and ITSupport must be contacted to request an account.

Initial DCC iRODS setup

The iRODS “i” commands are available from the HPC cluster nodes via the module command. Initial setup commands need to be run from a HPC node:

module load irods
iinit

The iinit command will prompt for your DTU password. Now you can issue icommands without having to enter your password.

Data transfer to/from iRODS should be done from a cluster node or transfer.gbar.dtu.dk and not one of the login-servers. Also large amounts of data should be stored on  /SCRATCH/<username> or /work1/<username> and not your home directory.

iCommands

  • Transferring data to/from iRODS

 $ imkdir test
 $ iput test1
 $ ils
 /dcc/home/iali/test:
 test1
 $ cd /SCRATCH/iali
 $ iget test1

  • To Transfer Multiple Files to iRODS

$ irsync -r . i:test2
$ ils
/dcc/home/iali:
C- /dcc/home/iali/test
C- /dcc/home/iali/test2
$ icd test2
$ ils
/dcc/home/iali/test2:
test1 test10 test2 test3  test4 test5 test6  test7  test8 test9

The following will sync data from the local test1 directory to the iRODS server

 irsync -r test1 i:test2

The following command will transfer from the iRODS test1 directory to the local test2

  irsync -r i:test1 test2

The i: indicates the iRODS server. Omitting the destination directory when copying to iRODS will create it.

irsync -r irods-test  i:

Using . when syncing from iRODS, it will create the directory in the current working directory:

 irsync -r i:irods-test .

Metadata Tagging

Basic syntax for adding meta data is

imeta <cmd> Filename AttName AttValue AttUnits

The following example uses:

AttName=name
AttValue=ian
AttUnits=text

imeta add -d irods-metadata name ian text
imeta ls  -d irods-metadata
imeta rm  -d irods-metadata name ian text
imeta ls  -d irods-metadata
$ imeta add -d irods-metadata name ian text

$ imeta ls  -d irods-metadata
AVUs defined for dataObj irods-metadata:
attribute: name
value: ian
units: text

$ imeta rm  -d irods-metadata name ian text
$ imeta ls  -d irods-metadata
AVUs defined for dataObj irods-metadata:
None

Searching by Metadata

Data from different project/program runs may be saved in multiple directories, but if tagged with Name, Run date etc. you can search on a tag name and value to find all such data.

$ imeta qu -d name = Lissimore
collection: /dcc/home/iali
dataObj: irods-metadata2
----
collection: /dcc/home/iali/testing
dataObj: irods-metadata3

$ imeta qu -d name = Ian      
collection: /dcc/home/iali
dataObj: irods-metadata

Sharing Data and Access Control

Using the ichown  command you can grant other users access to your iRODS data. To give an additional user complete write  to everything under directory run the following:

ichmod -r  write  iRODSUser directory/

My iRODS directory with initial permissions only for myself:

$ ils -Ar
/dcc/home/iali:
        ACL - iali#dcc:own   
        Inheritance - Disabled
  irods-metadata
        ACL - iali#dcc:own   
  irods-metadata2
        ACL - iali#dcc:own   
  C- /dcc/home/iali/testing
/dcc/home/iali/testing:
        ACL - iali#dcc:own   
        Inheritance - Disabled
  acl-test
        ACL - iali#dcc:own   
  irods-metadata3
        ACL - iali#dcc:own

Add additional iRODS user to have write access to the testing sub directory:

$ ichmod -r  write  user2 testing/
$ ils -Ar
/dcc/home/iali:
        ACL - iali#dcc:own   
        Inheritance - Disabled
  irods-metadata
        ACL - iali#dcc:own   
  irods-metadata2
        ACL - iali#dcc:own   
  C- /dcc/home/iali/testing
/dcc/home/iali/testing:
        ACL - user2#dcc:modify object   iali#dcc:own   
        Inheritance - Disabled
  acl-test
        ACL - user2#dcc:modify object   iali#dcc:own   
  irods-metadata3
        ACL - user2#dcc:modify object   iali#dcc:own

Getting Help for iCommands

ihelp iCommand