Skip to content

Export and Import of Projects, update and refactor for cli and drf #1394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 29 commits into
base: 2.3.3
Choose a base branch
from

Conversation

MyPyDavid
Copy link
Member

@MyPyDavid MyPyDavid commented Jul 11, 2025

Description

This PR updates and refactors the project export management command and adds a project import command.
Initially, It was GPT generated because many parts of the RDMO import functions were not easily re-usable for a management command. However the code has been re-evaluated and re-factored to fit better in the rdmo code base.
An extra added functionality is, the export and import of the project members (--include-memberships), it currently creates an extra JSON file with the memberships but it probably should be integrated to the export format (like XML). In another instance the users can then be created or retrieved via email addresses.
As a side note, for when using a socialaccount provider, that would require an extra "existing account adapter" that lets a social login be connected to the existing user via the verified email address for example.

This PR should aim to refactor the RDMO project export and import functions so that they can easily be re-used for these CLI commands and for the DRF endpoints.

CLI, management commands

The import_projects command:

--path DIR            Directory with exported project sub-folders (default: exports/).
--projects [ID ...]   Only import the listed project-ID folders.
--format FORMAT       Import plugin key to use (must be configured in ``settings.PROJECT_IMPORTS``).
--include-memberships 
[--create-new-users]       # optional: create users if they don't exist
[--as-user-id ID]          # request.user for plugin
[--as-username USERNAME]

and export_projects:

--export-mode {project,answers,view}   # default: project
--view-uri URI                         # required if --export-mode=view
--include-memberships                  # only valid in project mode
--projects [ID ...] --site-id --catalog-uri
--format xml|json|... --path DIR

Two methods render_member and render_member_user were added to the XML renderer.
The memberships can be exported to the xml and will look like this:

<memberships>
  <member>
    <role>owner|author|guest|…</role>
    <user>
      <id>…</id>
      <username>…</username>
      <first_name>…</first_name>
      <last_name>…</last_name>
      <full_name>…</full_name>
      <email>…</email>
    </user>
  </member>
  …
</memberships>

Try out locally:

# Export (project mode, XML, with memberships)
python manage.py export_projects \
  --export-mode project --include-memberships \
  --projects 1 --format xml --path /tmp/exports

# Import from exported directory (restore memberships, allow new users)
python manage.py import_projects \
  --dir /tmp/exports --projects 1 \
  --include-memberships --create-new-users

DRF, endpoints and actions

For exporting an existing project:

  • api/v1/projects/projects/<pk>/export/<export_format>

For project creation with an import

  • /api/v1/projects/projects/{id}/import-create-preview
  • /api/v1/projects/projects/{id}/import-create-confirm/

For updating an existing project with an import

  • /api/v1/projects/projects/{id}/import-update-preview
  • /api/v1/projects/projects/{id}/import-update-confirm/

Related issues are #1277 #1357

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
@MyPyDavid MyPyDavid self-assigned this Jul 11, 2025
@MyPyDavid MyPyDavid changed the title projects: update export and add import command Update and Refactor Export and Import of Projects Jul 11, 2025
@MyPyDavid MyPyDavid changed the title Update and Refactor Export and Import of Projects Update and Refactor Export and Import of Projects (for cli and drf) Jul 11, 2025
@MyPyDavid MyPyDavid changed the title Update and Refactor Export and Import of Projects (for cli and drf) Export and Import of Projects, update and refactor for cli and drf Jul 11, 2025
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
@MyPyDavid MyPyDavid linked an issue Jul 18, 2025 that may be closed by this pull request
MyPyDavid added 10 commits July 18, 2025 18:14
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
…and use django FileSystemStorage

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
@MyPyDavid
Copy link
Member Author

MyPyDavid commented Aug 14, 2025

  • add memberships to project export (xml (no dc-uri), etc..)
    • Memberships can be exported by all users that can export a project (names and emails are already visible in the project UI).
  • import projects with members can only be done by site managers (and higher)
  • potentially create (silent invite aka directly add membership) or 'lookup' users for the memberships?

Import with memberships is only supported via the management command for now.

Other notes:
What about the settings.USER_API is it still needed?

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use the API to download project exports
1 participant