|
| 1 | +# Production Files Access |
| 2 | + |
| 3 | +Starting from production campaign 25.01.1 onwards. Now files are registred to a catalog in Rucio (Data Mangement Software). |
| 4 | +All old files will be registred to rucio, notification to follow |
| 5 | + |
| 6 | +To see what files are available and how to access it use Rucio. |
| 7 | + |
| 8 | +- Files are now registered with Rucio, the Data Management System. |
| 9 | +- Access Rucio client from `eic-shell`: |
| 10 | + ```shell |
| 11 | + $ ./eic-shell |
| 12 | + $ rucio whoami |
| 13 | + ``` |
| 14 | + |
| 15 | +- Files are registered in Rucio with a format similar to xrootd, excluding the base path `/volatile/eic/EPIC`. |
| 16 | +- Data Identifiers (DID) are structured as `scope:name`. For "epic," the scope is always `epic`, resulting in DIDs like: |
| 17 | + - `epic:/EVGEN/...` for EVEGEN files |
| 18 | + - `epic:/FULL/...` for FULL simulation files |
| 19 | + - `epic:/RECO/...` for Reconstructed output files |
| 20 | + - `epic:/LOGS/...` for Log files |
| 21 | + |
| 22 | +## Step by step guide. |
| 23 | +### First find the location of files. |
| 24 | + |
| 25 | +- **List datasets for a campaign** : |
| 26 | + command: `rucio list-dids --short <dids>` |
| 27 | + ```shell |
| 28 | + $ rucio list-dids --short epic:/RECO/25.01.1/* |
| 29 | +
|
| 30 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/18x275/minQ2=100 |
| 31 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=1 |
| 32 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/18x275/minQ2=1000 |
| 33 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/18x275/minQ2=1 |
| 34 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10 |
| 35 | + epic:/RECO/25.01.1/epic_craterlake/BACKGROUNDS/SYNRAD/dataprod_rel_1.0.0/18x275 |
| 36 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/18x275/minQ2=10 |
| 37 | + .... |
| 38 | + ``` |
| 39 | + Note: The list shown above is truncated for preview. |
| 40 | + Replace `25.01.1` with any other campaign. |
| 41 | + |
| 42 | +- **List files within a dataset**: |
| 43 | + command: `rucio list-content --short <did>` |
| 44 | + ```shell |
| 45 | + $ rucio list-content --short epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10 |
| 46 | +
|
| 47 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1105.eicrecon.tree.edm4eic.root |
| 48 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1106.eicrecon.tree.edm4eic.root |
| 49 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1107.eicrecon.tree.edm4eic.root |
| 50 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1108.eicrecon.tree.edm4eic.root |
| 51 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1109.eicrecon.tree.edm4eic.root |
| 52 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1110.eicrecon.tree.edm4eic.root |
| 53 | + epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1111.eicrecon.tree.edm4eic.root |
| 54 | + ``` |
| 55 | + Note: The list shown above is truncated for preview. |
| 56 | + |
| 57 | +- **Find location of files**: |
| 58 | + command: `rucio list-file-replicas --protocol root --pfns <did>` |
| 59 | + ```shell |
| 60 | + $ rucio list-file-replicas --protocol root --pfns epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root |
| 61 | +
|
| 62 | + root://dtn-rucio.jlab.org:1094//volatile/eic/EPIC//RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root |
| 63 | + ``` |
| 64 | + |
| 65 | +#### Access methods for production files include: |
| 66 | + |
| 67 | +- **Download all files in a dataset at once with `rucio download <dataset>`** |
| 68 | + ```shell |
| 69 | + rucio download epic:/RECO/25.01.1/epic_craterlake/DIS/NC/18x275/minQ2=100 |
| 70 | + ``` |
| 71 | + |
| 72 | + or multiple datasets at once |
| 73 | + ```shell |
| 74 | + rucio download <dataset1> <dataset2> |
| 75 | + ``` |
| 76 | + |
| 77 | +- **Download single file with `rucio download <file_did>`** |
| 78 | + ```shell |
| 79 | + rucio download epic:/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.1105.eicrecon.tree.edm4eic.root |
| 80 | + ``` |
| 81 | + |
| 82 | +- **Copying with `xrdcp`:** |
| 83 | + ```shell |
| 84 | + xrdcp root://dtn-rucio.jlab.org:1094//volatile/eic/EPIC/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root <local destination> |
| 85 | + ``` |
| 86 | + |
| 87 | +- **Copying with `gfal-copy`:** |
| 88 | + ```shell |
| 89 | + gfal-copy root://dtn-rucio.jlab.org:1094//volatile/eic/EPIC/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root <local destination> |
| 90 | + ``` |
| 91 | + |
| 92 | +- **Opening directly with ROOT:** |
| 93 | + ```c++ |
| 94 | + auto f = TFile::Open("root://dtn-rucio.jlab.org:1094//volatile/eic/EPIC/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root") |
| 95 | + ``` |
| 96 | + or using Python |
| 97 | + ```python |
| 98 | + import uproot |
| 99 | + file_path = "root://dtn-rucio.jlab.org:1094//volatile/eic/EPIC/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root" |
| 100 | + root_file = uproot.open(file_path) |
| 101 | + ``` |
| 102 | + |
| 103 | + ```python |
| 104 | + import ROOT |
| 105 | + file_path = "root://dtn-rucio.jlab.org:1094//volatile/eic/EPIC/RECO/25.01.1/epic_craterlake/DIS/NC/10x100/minQ2=10/pythia8NCDIS_10x100_minQ2=10_beamEffects_xAngle=-0.025_hiDiv_5.0255.eicrecon.tree.edm4eic.root" |
| 106 | + file = ROOT.TFile.Open(file_path, "READ") |
| 107 | + ``` |
| 108 | + |
0 commit comments