You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/development.md
+25-12Lines changed: 25 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,7 +51,7 @@ Follow the steps below to configure the necessary settings.
51
51
NMDC_SESSION_SECRET_KEY=changeme
52
52
NMDC_API_JWT_SECRET=changeme
53
53
```
54
-
54
+
55
55
### NERSC Credentials
56
56
57
57
To load production data or to run an ingest locally, you will need NERSC credentials.
@@ -78,13 +78,13 @@ Google Cloud Storage (GCS) is used to store images associated with Submission Po
78
78
79
79
Whether you use the real or fake GCS server, you will need to set up authentication. The recommended way to do this is to use Application Default Credentials (ADC) with service account impersonation. Service account impersonation is required for generating signed URLs for uploading/downloading images directly to/from GCS.
80
80
81
-
1. Ask a team member with the necessary GCS permissions to associate your Google Cloud account with the NMDC Google Cloud project and service account.
81
+
1. Ask a team member with the necessary GCS permissions to associate your Google Cloud account with the NMDC Google Cloud project and service account.
82
82
2. Install the Google Cloud Command Line Interface (CLI) by following the instructions at https://cloud.google.com/sdk/docs/install.
83
83
3. Run the following command to set up Application Default Credentials (ADC):
84
84
```bash
85
85
gcloud auth application-default login --impersonate-service-account <service account email will be provided by team member>
86
86
```
87
-
87
+
88
88
You also must generate a local object name prefix. This prefix is used to differentiate which system uploaded to the shared GCS bucket. Local development systems should use the prefix `local_<random_suffix>`.
89
89
90
90
1. Generate a random suffix using the following command:
@@ -166,7 +166,7 @@ Although the project is designed to be run in Docker, having the dependencies in
166
166
source .venv/bin/activate
167
167
pip install -e .
168
168
```
169
-
169
+
170
170
### Frontend dependencies
171
171
172
172
1. Install the frontend dependencies.
@@ -182,7 +182,7 @@ Run the full stack via Docker Compose:
182
182
<!-- TODO: Consider adding `--build` to this command so that Docker Compose builds
183
183
the containers, rather than pulling from from GHCR (unless you
184
184
want to use the versions that happen to currently be on GHCR).
185
-
This has to do with the fact that the `docker-compose.yml` file
185
+
This has to do with the fact that the `docker-compose.yml` file
186
186
contains service specs having both an `image` and `build` section. -->
187
187
188
188
```bash
@@ -220,16 +220,16 @@ yarn serve
220
220
<details>
221
221
<summary>Running yarn via npx?</summary>
222
222
223
-
When you run `$ npx yarn serve`while using a Node.js version newer than 17, the frontend development server may fail to start and may, instead, display the error code "`ERR_OSSL_EVP_UNSUPPORTED`".
223
+
When you run `$ npx yarn serve`while using a Node.js version newer than 17, the frontend development server may fail to start and may, instead, display the error code "`ERR_OSSL_EVP_UNSUPPORTED`".
224
224
225
-
You can work around that error by prefixing the command with "`NODE_OPTIONS=--openssl-legacy-provider`", as explained [here](https://stackoverflow.com/a/70582385) and shown below:
225
+
You can work around that error by prefixing the command with "`NODE_OPTIONS=--openssl-legacy-provider`", as explained [here](https://stackoverflow.com/a/70582385) and shown below:
View the main application at `http://127.0.0.1:8081/`. Changes to files in the `web` directory will automatically trigger a reload in your browser.
232
+
View the main application at `http://127.0.0.1:8081/`. Changes to files in the `web` directory will automatically trigger a reload in your browser.
233
233
234
234
>**Note**: An instance of the frontend application will continue to be served via Docker Compose on port `8080`, but that instance will not pick up changes to the `web` directory automatically. Be aware of which port you are accessing when doing frontend development.
235
235
@@ -241,7 +241,7 @@ It is recommended to use `127.0.0.1` instead of `localhost` for local developmen
241
241
242
242
>**Note**: This is not generally required unless you are specifically working on the ingest code. If you are working on the web application, simply [loading from a recent production backup](#load-production-data) is sufficient.
243
243
244
-
1. Ensure that you have completed the sections above about configuring your [NERSC credentials](#nersc-credentials) and [MongoDB credentials](#mongodb-credentials).
244
+
1. Ensure that you have completed the sections above about configuring your [NERSC credentials](#nersc-credentials) and [MongoDB credentials](#mongodb-credentials).
245
245
2. Obtain a new SSH key from NERSC if you haven't done so in the last 24 hours.
246
246
```bash
247
247
sshproxy.sh -u $NERSC_USER
@@ -258,8 +258,8 @@ It is recommended to use `127.0.0.1` instead of `localhost` for local developmen
258
258
-i ~/.ssh/nersc \
259
259
dtn01.nersc.gov
260
260
```
261
-
> That command will set up SSH port forwarding such that your computer can access the dev MongoDB server at `localhost:37018` and the prod MongoDB server at `localhost:37019`.
262
-
261
+
> That command will set up SSH port forwarding such that your computer can access the dev MongoDB server at `localhost:37018` and the prod MongoDB server at `localhost:37019`.
262
+
263
263
> From within a Docker container `host.docker.internal` can be used to access the `localhost` of your computer. When ingesting from the dev or prod MongoDB instances, be sure to set `NMDC_MONGO_HOST=host.docker.internal` in your `.env` file.
264
264
265
265
> See https://github.com/microbiomedata/infra-admin/blob/main/mongodb/connection-guide.md (internal) for more information on connecting to the MongoDB instances.
@@ -275,7 +275,7 @@ It is recommended to use `127.0.0.1` instead of `localhost` for local developmen
275
275
```bash
276
276
docker-compose run backend nmdc-server ingest -vv --function-limit 100
277
277
```
278
-
278
+
279
279
> **Note**: The `--function-limit` flag is optional. It is used to reduce the time that the ingest takes by limiting the number of certain types of objects loaded. This can be useful for testing purposes. For more information on options run `nmdc-server ingest --help`.
280
280
281
281
## Testing
@@ -284,6 +284,19 @@ It is recommended to use `127.0.0.1` instead of `localhost` for local developmen
284
284
tox
285
285
```
286
286
287
+
In order for the `py312` test suite to run properly, it needs to be able to communicate with a running `postgres` server and the fake GCS service. You can use the docker configuration to get these services up and running:
288
+
289
+
```bash
290
+
docker compose up db storage
291
+
```
292
+
293
+
You'll also need to set environment variables so the tests know where these resources can be found. If you're running the services via `docker compose`, then you can use the following values:
0 commit comments