Stash: Temporary Data Storage

Overview

Stash runs on Ceph, a highly-scalable, clustered, POSIX-compliant network filesystem. You may use this feature for temporary data storage.

This page will introduce users to accessing data stored on stash remotely using HTTP and XRootD.

Stash Area

After you login to login-el7.uscms.org, you can access your stash area as followed:

Stash is mounted on the CMS Connect login node ( login-el7.uscms.org) at /stash, and is accessible to workers via XRootD. The portion of your Stash designated public is also available directly via HTTP.

Set up tutorial files

% ssh username@login-el7.uscms.org

$ ls /stash/user/khurtado

public work

$ cd /stash/user/khurtado/work

$ ls

test .txt

Accessing Stash via HTTP

User accounts on CMS Connect have a directory that is automatically web accessible via http://stash.osgconnect.net/+username . This directory is located at /stash2/user/username/public. To make a file or directory accessible, copy it to this directory or a subdirectory of this directory and give files permissions of 644 and directories permissions of 755. E.g. :

 

stash.png

Browsing the public Stash area

Copying files via HTTP

You can download your files using wget easily:

Accessing files using wget

$ wget --no-check-certificate http://stash.osgconnect.net/+khurtado/rootlogon.C

--2015-10-30 15:56:18-- http://stash.osgconnect.net/+khurtado/rootlogon.C

Resolving stash.osgconnect.net... 192.170.227.183

=Connecting to stash.osgconnect.net

192.170.227.183 :80... connected.=

HTTP request sent, awaiting response... 302 Found

Location: http://stash.osgconnect.net/+khurtado/rootlogon.C [following]

--2015-10-30 15:56:18-- http://stash.osgconnect.net/+khurtado/rootlogon.C

Resolving stash.osgconnect.net... 192.170.227.183

Reusing existing connection to stash.osgconnect.net:80.

HTTP request sent, awaiting response... 200 OK

Length: 3996 (3.9K) [text/x-c]

Saving to: “rootlogon.C”

100%[===================================================================================================================================================>] 3,996 --.-K/s in 0s

2015-10-30 15:56:19 (176 MB/s) - “rootlogon.C” saved [3996/3996]

You can also use curl, for example:

Accessing files using wget

$ curl -L http://stash.osgconnect.net/+khurtado/rootlogon.C > rootlogon.C

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

102 3996 102 3996   0    0  234k     0 --:--:-- --:--:-- --:--:-- 234k

Access Data via XRootD

Using the XRootD service (AAA) it is possible to read data stored in e.g Tier-2 Sites, CERN, Fermilab or to write to such sites if you have write permissions to do so. Please, read Section 5.13 of the CMS Offline Workbook for full instructions on how to either copy files via xrdcp or to directly open files via AAA using in CMSSW and ROOT.

Notice that in order to use this service from the CMS Connect login node you will need to:

  • Setup a release of the CMSSW framework.
  • Have and initialize a valid proxy certificate.
Here is one example on how to access a file (having the Logical Filename) via AAA:

On ROOT:

ROOT

TFile *f TFile::Open( ="root://cmsxrootd.fnal.gov///store/mc/SAM/GenericTTbar/GEN-SIM-RECO/CMSSW_5_3_1_START53_V5-v1/0013/CE4D66EB-5AAE-E111-96D6-003048D37524.root" );

Otherwise, you can use xrdcp to copy your file:

xrdcp root://cmsxrootd.fnal.gov///store/mc/SAM/GenericTTbar/GEN-SIM-RECO/CMSSW_5_3_1_START53_V5-v1/0013/CE4D66EB-5AAE-E111-96D6-003048D37524.root foo.root

-- CarlLundstedt - 2022-12-01

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2022-12-01 - CarlLundstedt
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback