Data Services Guide
Tehdyt toimenpiteet

Using the data services at CSC

Release 1.0 Mar. 2010

Contents

What's new
Open issues
Preface

1. Introduction
   1.1 Notation used in this guide
   1.2 Data services in the computing environment of CSC
   1.3 Getting access to the resources
   1.4 Connecting to the servers of CSC
   1.5 Managing your CSC user account

2. Linux basics for CSC
   2.1 Using linux in command line
   2.2 Text and image processing
   2.3 Working with tcsh shell

3. Linux tools for working with data
   3.1 Selecting rows from a file
   3.2 Selecting columns from a file
   3.3 Sorting the data and removing redundancy
   3.4 Replacing characters and strings

4. Directories at CSC
   4.1 Monitoring the disk usage
   4.2 Home directory ($HOME)
   4.3 Temporary directories ($WRKDIR, $METAWRK, $TMPDIR)
   4.4 Archive directory ($ARCHIVE)
   4.5 Software installation directory ($USERAPPL)
   4.6 Project directories

5. Managing files at CSC systems
   5.1 Finding and studying files
   5.2 Packing and compression tools
   5.3 Managing access permissions of files and directories
   5.4 Using checksums to verify successful data storage and transfer
   5.6 Linux scripts

6. Moving data between CSC and local environment
   6.1 Data transport with Scientist's User Interface
   6.2 Copying files from linux and Mac OSX machines with scp
   6.3 File transfer in Windows with WinSCP
   6.4 Using rsync for data transfer and synchronization
   6.5 Using wget to download data from web sites to CSC
   6.6 Remote disk mounts

7. Relational database service at kaivos.csc.fi
   7.0 How to became a CSC database user?

   7.1 MySQL client programs in Hippu and Murska
   7.2 Data import and export
   7.3 Using MySQL client through batch job system
   7.4 Using kaivos.csc.fi directly from your local computer
   7.5 Using MySQL perl API at CSC
   7.6 Using MySQL python API at CSC
   7.7 Using MatLab database toolbox

Appendix
   A. Linux commands discussed in this guide