iRODS is a data management platform designed to meet the requirements of organizations in handling massive amounts of data by consolidating heterogenous storage resources under a single unified namespace and providing seamless and interactive access to its content.
iRODS offers multiple options for users to manage their data stored in the facility such as the native iRODS “i-commands” or via the HTTP/S WebDAV protocol.
TERMINOLOGY Resource - a storage system managed by iRODS. Object - a file stored in iRODS. Collection - is equivalent to directories in *nix file systems.
A. iRODS iCommands (For Linux OS)¶
The icommands is a collection of command line utilities/program used by users to interact with CoARE’s data storage service. This is the recommended method for large file (2GB and above in size) and bulk data (many small files) transfers. The icommands by default, invokes multiple threads to perform parallel file transfer operations.
- Download the appropriate packages for your operating system from http://irods.org/download/. Scroll down to the “iCommands CLI” section to find the links to the packages.
- Follow the instructions prescribed by the operating system to install the packages
- Download the irods configuration file from http://irods.asti.dost.gov.ph/irods_environment.json and save it into the ~/.irods/ directory.
- No modification is necessary for anonymous access. For CoARE project users, kindly visit this page for more information on how to access your personal or project collections in CoARE iRODS.
- Setup your connection to CoARE iRODS server. This command will cache the password into a file so that all subsequent icommand operations will not require user password input.
- Start exploring the public data collections
- iRODS collections are file structures that are similar to directories in Unix file systems. To navigate from one collection to another, use the ‘icd’ command. To list the content, use ‘ils’.
- Users are referred to iRODS online wiki for a more comprehensive discussion on the usage of icommands (https://docs.irods.org/4.1.5/).
- Anonymous users have limited file operation privileges on the server. Download and read operations are only enabled.
- This command will download a whole collection from the server. The ‘-r’ option specifies to do a recursive transfer, i.e. to include the sub-collections under a collection:
$ iget -rvfP [name of collection]
- This command prints out more information on the usage of iget:
$ iget -h
Useful Commands (View list of iCommands)
B. WebDAV (For Windows, Mac, & Linux OS)¶
WebDAV is an extension to the popular HTTP protocol that allows client to perform remote authoring operations on the server. While it supports a broad selection of operating systems, this method is comparably slower than the icommands and is only recommended for small file transfer operations.
Linux OS: Nautilus File Manager¶Nautilus File Manager in Linux has native support for connecting to WebDAV resources. To access iRODS using Nautilus, follow the steps below:
- Open Nautilus
- click on the Connect to Server link at the lefthand side of the windows
- Enter the details needed then click Connect
- Service Type: Custom Location
- Location: https://irods-webdav.asti.gov.ph/users
- Input any password when prompted. The server will issue an error message if no password is supplied
- Navigate iRODS and data stored there.
Mac and Windows OS: CyberDUck¶CyberDuck is the most compatible webdav client we have tested to interface with the CoARE data storage service. It is an open-source file transfer client that supports various protocols such as FTP/S, SSH, OpenStack Swift, Amazon S3 and WebDAV.
- Download and install the latest package of CyberDuck from:
- Download the pre-configured CyberDuck configuration profile from:
- Click on the file to load it as a bookmark in CyberDuck. This serves as a shortcut for connecting to the public collection of the data storage facility.
- Users may encounter an alert message regarding the validity of the certificate. Just click on the Always trust checkbox and proceed by hitting Continue.
- The native iRODS protocol (used in icommands operations) passes data and control communications through port 1247. Seek the assistance of your local network administrator and request to adjust the firewall policies to permit client connections to the server through the specified port.
- The collections in the iRODS data storage facility may contain tremendous number of files which may not appear immediately when accessed through the client interface. It may take an average of one minute for the listing to come up. YMMV.