Skip to content

Commit fa1cb7d

Browse files
winromuluswinromulus
committed
Initial implementation of the SFTP service (#1)
- Added SFTP docker image - Added helm chart - Added management scripts - Added Azure pipelines integration
1 parent 46042b4 commit fa1cb7d

25 files changed

+1436
-2
lines changed

‎README.md

Lines changed: 167 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,167 @@
1-
# ES.Docker.SFTP
2-
SFTP Server for Docker
1+
# SFTP ([SSH File Transfer Protocol](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol)) server using [OpenSSH](https://en.wikipedia.org/wiki/OpenSSH)
2+
This project provides a Docker image for hosting a SFTP server. Included are `Docker` (`docker-cli` and `docker-compose`) and `Kubernetes` (`kubectl` and `helm`) deployment scripts
3+
4+
[![Build Status](https://dev.azure.com/emberstack/OpenSource/_apis/build/status/docker-sftp?branchName=master)](https://dev.azure.com/emberstack/OpenSource/_build/latest?definitionId=16&branchName=master)
5+
[![Release](https://img.shields.io/github/release/emberstack/docker-sftp.svg?style=flat-square)](https://github.com/emberstack/docker-sftp/releases/latest)
6+
[![GitHub Tag](https://img.shields.io/github/tag/emberstack/docker-sftp.svg?style=flat-square)](https://github.com/emberstack/docker-sftp/releases/latest)
7+
[![Docker Image](https://images.microbadger.com/badges/image/emberstack/sftp.svg)](https://microbadger.com/images/emberstack/sftp)
8+
[![Docker Version](https://images.microbadger.com/badges/version/emberstack/sftp.svg)](https://microbadger.com/images/emberstack/sftp)
9+
[![Docker Pulls](https://img.shields.io/docker/pulls/emberstack/sftp.svg?style=flat-square)](https://hub.docker.com/r/emberstack/sftp)
10+
[![Docker Stars](https://img.shields.io/docker/stars/emberstack/sftp.svg?style=flat-square)](https://hub.docker.com/r/remberstack/sftp)
11+
[![license](https://img.shields.io/github/license/emberstack/docker-sftp.svg?style=flat-square)](LICENSE)
12+
13+
14+
> Supports architectures: `amd64`. Coming soon: `arm` and `arm64`
15+
16+
## Usage
17+
18+
The SFTP server can be easily deployed to any platform that can host containers based on Docker.
19+
Below are deployment methods for:
20+
- Docker CLI
21+
- Docker-Compose
22+
- Kubernetes using Helm (recommended for Kubernetes)
23+
- Kubernetes (manual)
24+
25+
Process:
26+
1) Create server configuration
27+
2) Mount volumes as needed
28+
3) Set host file for consistent server fingerprint
29+
30+
### Configuration
31+
32+
The SFTP server uses a `json` based configuration file for default server options and to define users. This file has to be mounted on `/sftp/config/sftp.json` inside the container.
33+
Environment variable based configuration is not supported (see the `Advanced Configuration` section below for the reasons).
34+
35+
Below is the simplest configuration file for the SFTP server:
36+
37+
```json
38+
{
39+
"global": {
40+
"chroot": {
41+
"directory": "%h",
42+
"startPath": "sftp"
43+
},
44+
"directories": [ "sftp" ]
45+
},
46+
"users": [
47+
{
48+
"username": "demo",
49+
"password": "password"
50+
},
51+
{
52+
"username": "demo2",
53+
"password": "password"
54+
}
55+
]
56+
}
57+
```
58+
This configuration creates a user `demo` with the password `demo`.
59+
A directory "sftp" is created for each user in the own home and is accessible for read/write.
60+
The user is `chrooted` to the `/home/demo` directory. Upon connect, the start directory is `sftp`.
61+
62+
You can add additional users, default directories or customize start directories per user. You can also define the `uid` and `gid` for each user. See the `Advanced Configuration` section below for all configuration options.
63+
64+
65+
### Deployment using Docker CLI
66+
67+
> Simple Docker CLI run
68+
69+
```shellsession
70+
$ docker run -p 22:22 -d emberstack/sftp --name sftp
71+
```
72+
This will start a SFTP in the container `sftp` with the default configuration. You can connect to it and login with the `user: demo` and `password: demo`.
73+
74+
> Provide your configuration
75+
76+
```shellsession
77+
$ docker run -p 22:22 -d emberstack/sftp --name sftp -v /host/sftp.conf:/sftp/config/sftp.conf:ro
78+
```
79+
This will override the default (`/sftp/config/sftp.conf`) configuration with the one from the host `/host/sftp.conf`.
80+
81+
> Mount a directory from the host for the user 'demo'
82+
83+
```shellsession
84+
$ docker run -p 22:22 -d emberstack/sftp --name sftp -v /host/sftp.conf:/sftp/config/sftp.conf:ro -v /host/demo:/home/demo/sftp
85+
```
86+
This will mount the `demo` directory from the host on the `sftp` directory for the "demo" user.
87+
88+
89+
### Deployment using Docker Compose
90+
91+
> Simple docker-compose configuration
92+
93+
Create a docker-compose configuration file:
94+
```yaml
95+
version: '3'
96+
services:
97+
sftp:
98+
image: "emberstack/sftp"
99+
ports:
100+
- "22:22"
101+
volumes:
102+
- ../config-samples/sample.sftp.json:/sftp/config/sftp.json:ro
103+
```
104+
And run it using docker-compose
105+
```shellsession
106+
$ docker-compose -p sftp -f docker-compose.yaml up -d
107+
```
108+
109+
The above configuration is available in the `deploy\docker-compose` folder in this repository. You can use it to start customizing the deployment for your environment.
110+
111+
112+
113+
### Deployment to Kubernetes using Helm
114+
115+
Use Helm to install the latest released chart:
116+
```shellsession
117+
$ helm repo add emberstack https://emberstack.github.io/helm-charts
118+
$ helm repo update
119+
$ helm upgrade --install sftp emberstack/sftp
120+
```
121+
122+
You can customize the values of the helm deployment by using the following Values:
123+
124+
| Parameter | Description | Default |
125+
| ------------------------------------ | -------------------------------------------------------------------------------- | ------------------------------------------------------- |
126+
| `nameOverride` | Overrides release name | `""` |
127+
| `fullnameOverride` | Overrides release fullname | `""` |
128+
| `image.repository` | Container image repository | `emberstack/sftp` |
129+
| `image.tag` | Container image tag | `latest` |
130+
| `image.pullPolicy` | Container image pull policy | `Always` if `image.tag` is `latest`, else `IfNotPresent`|
131+
| `storage.volumes` | Defines additional volumes for the pod | `{}` |
132+
| `storage.volumeMounts` | Defines additional volumes mounts for the sftp container | `{}` |
133+
| `configuration.global.chroot.directory` | Global chroot directory for the `sftp` user group. Can be overriden per-user | `"%h"` |
134+
| `configuration.global.chroot.startPath` | Start path for the `sftp` user group. Can be overriden per-user | `"sftp"` |
135+
| `configuration.global.directories` | Directories that get created for all `sftp` users. Can be appended per user | `["sftp"]` |
136+
| `configuration.users` | Array of users and their properties | Contains `demo` user by default |
137+
| `configuration.users[].username` | Set the user's username | N/A |
138+
| `configuration.users[].password` | Set the user's password. If empty or `null`, password authentication is disabled | N/A |
139+
| `configuration.users[].passwordEncrypted` | `true` or `false`. Indicates if the password value is already encrypted | `false` |
140+
| `configuration.users[].passwordEncrypted` | `true` or `false`. Indicates if the password value is already encrypted | `false` |
141+
| `configuration.users[].chroot` | If set, will override global `chroot` settings for this user. | `null` |
142+
| `configuration.users[].directories` | Array of additional directories created for this user | `null` |
143+
| `initContainers` | Additional initContainers for the pod | `{}` |
144+
| `resources` | Resource limits | `{}` |
145+
| `nodeSelector` | Node labels for pod assignment | `{}` |
146+
| `tolerations` | Toleration labels for pod assignment | `[]` |
147+
| `affinity` | Node affinity for pod assignment | `{}` |
148+
149+
> Find us on [Helm Hub](https://hub.helm.sh/charts/emberstack)
150+
151+
152+
### Deployment to Kubernetes using kubectl
153+
Each release (found on the [Releases](https://github.com/EmberStack/docker-sftp/releases) GitHub page) contains the manual deployment file (`sftp.yaml`).
154+
155+
```shellsession
156+
$ kubectl apply -f https://github.com/EmberStack/docker-sftp/releases/latest/download/sftp.yaml
157+
```
158+
159+
## Advanced Configuration
160+
161+
TODO: This section is under development due to the number of configuration options being added. Please open an issue on the [emberstack/docker-sftp](https://github.com/emberstack/docker-sftp) project if you need help.
162+
163+
164+
165+
## Final Word
166+
This project is a work in progress. More features and configuration options will be added. If you want to contribute or have feedback, please feel free to create an Issue or contrinute with a Pull Request.
167+
This project was initially inspired by [atmoz/sftp](https://github.com/atmoz/sftp) but has changed from the original design to increase flexibility and range of supported deployment options. This is no longer a lightweight SFTP server, the target being a rich set of features, to the detriment of simplicity. If you're looking for a simple SFTP docker image, please consider the [atmoz/sftp](https://github.com/atmoz/sftp) project.

‎azure-pipelines.yaml

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
name: $(coreVersion).$(Date:yy)$(DayOfYear).$(Rev:r)
2+
3+
variables:
4+
coreVersion: 1
5+
buildConfiguration: "Release"
6+
imageRepository: "emberstack/sftp"
7+
DOCKER_CLI_EXPERIMENTAL: 'enabled'
8+
9+
trigger:
10+
branches:
11+
include:
12+
- "*"
13+
14+
stages:
15+
16+
- stage: build
17+
displayName: "Build"
18+
jobs:
19+
- job: build
20+
displayName: "Build Job"
21+
pool:
22+
vmImage: "Ubuntu-16.04"
23+
steps:
24+
- script: |
25+
mkdir -p artifacts/helm
26+
mkdir -p artifacts/kubectl
27+
workingDirectory: '$(Build.ArtifactStagingDirectory)'
28+
displayName: 'Create Artifacts directories'
29+
30+
- task: HelmInstaller@0
31+
displayName: 'Install Helm'
32+
inputs:
33+
helmVersion: 2.13.1
34+
kubectlVersion: 1.10.11
35+
36+
- script: 'helm lint'
37+
workingDirectory: deploy/helm/sftp
38+
displayName: 'helm lint'
39+
40+
- task: HelmDeploy@0
41+
displayName: 'helm package'
42+
inputs:
43+
connectionType: None
44+
command: package
45+
chartPath: deploy/helm/sftp
46+
chartVersion: '$(Build.BuildNumber)'
47+
destination: '$(Build.ArtifactStagingDirectory)/artifacts/helm'
48+
49+
- script: 'helm template --name sftp sftp > $(Build.ArtifactStagingDirectory)/artifacts/kubectl/sftp-$(Build.BuildNumber).yaml'
50+
workingDirectory: deploy/helm
51+
displayName: 'helm template'
52+
53+
- publish: '$(Build.ArtifactStagingDirectory)/artifacts/helm'
54+
artifact: 'helm'
55+
56+
- publish: '$(Build.ArtifactStagingDirectory)/artifacts/kubectl'
57+
artifact: 'kubectl'
58+
59+
- task: Docker@2
60+
displayName: "Build amd64 image"
61+
inputs:
62+
containerRegistry: "Emberstack Docker Hub"
63+
repository: $(imageRepository)
64+
Dockerfile: src/Dockerfile
65+
command: build
66+
buildContext: src
67+
tags: "build-$(Build.BuildNumber)-amd64"
68+
69+
- task: Docker@2
70+
displayName: "Push images"
71+
inputs:
72+
containerRegistry: "Emberstack Docker Hub"
73+
repository: $(imageRepository)
74+
command: push
75+
tags: |
76+
build-$(Build.BuildNumber)-amd64
77+
78+
- task: Docker@2
79+
displayName: "Docker Hub Login"
80+
inputs:
81+
containerRegistry: "Emberstack Docker Hub"
82+
command: login
83+
84+
- script: |
85+
docker manifest create $(imageRepository):build-$(Build.BuildNumber) $(imageRepository):build-$(Build.BuildNumber)-amd64
86+
docker manifest inspect $(imageRepository):build-$(Build.BuildNumber)
87+
docker manifest push $(imageRepository):build-$(Build.BuildNumber)
88+
displayName: "Create and push multi-arch manifest"
89+
90+
91+
- stage: release
92+
displayName: "Release"
93+
dependsOn: 'build'
94+
condition: and(succeeded(), in(variables['Build.Reason'], 'IndividualCI', 'Manual'), in(variables['Build.SourceBranchName'], 'master'))
95+
jobs:
96+
- job: release
97+
displayName: "Release Job"
98+
pool:
99+
vmImage: "Ubuntu-16.04"
100+
variables:
101+
- group: "OpenSource.GitHub"
102+
steps:
103+
- checkout: none
104+
105+
- download: current
106+
artifact: 'helm'
107+
108+
- download: current
109+
artifact: 'kubectl'
110+
111+
- task: Docker@2
112+
displayName: "Docker Login"
113+
inputs:
114+
containerRegistry: "Emberstack Docker Hub"
115+
command: login
116+
117+
- task: HelmInstaller@0
118+
displayName: 'Install Helm'
119+
inputs:
120+
helmVersion: 2.13.1
121+
kubectlVersion: 1.10.11
122+
123+
- script: |
124+
docker pull $(imageRepository):build-$(Build.BuildNumber)-amd64
125+
docker manifest create $(imageRepository):release-$(Build.BuildNumber) $(imageRepository):build-$(Build.BuildNumber)-amd64
126+
docker manifest push $(imageRepository):release-$(Build.BuildNumber)
127+
docker manifest create $(imageRepository):latest $(imageRepository):build-$(Build.BuildNumber)-amd64
128+
docker manifest push $(imageRepository):latest
129+
displayName: 'docker pull, tag and push'
130+
131+
- script: |
132+
git config --global user.email "$(emberstack-agent-email)"
133+
git config --global user.name "$(emberstack-agent-name)"
134+
git clone https://$(emberstack-agent-username):$(emberstack-agent-pat)@github.com/EmberStack/helm-charts.git
135+
136+
mkdir -p helm-charts/charts
137+
cp $(Pipeline.Workspace)/helm/sftp-$(Build.BuildNumber).tgz helm-charts/charts
138+
139+
cd helm-charts
140+
rm index.yaml
141+
helm repo index ./
142+
git add .
143+
git status
144+
git commit -m "Added sftp-$(Build.BuildNumber).tgz"
145+
git push
146+
displayName: 'Add chart to GitHub repository'
147+
148+
- script: |
149+
mkdir -p github
150+
cp $(Pipeline.Workspace)/kubectl/sftp-$(Build.BuildNumber).yaml github/sftp.yaml
151+
displayName: 'Prepare GitHub Artifacts'
152+
153+
- task: GitHubRelease@1
154+
displayName: 'GitHub release (create)'
155+
inputs:
156+
gitHubConnection: GitHub
157+
repositoryName: 'EmberStack/docker-sftp'
158+
tagSource: userSpecifiedTag
159+
tag: 'v$(Build.BuildNumber)'
160+
title: 'Release v$(Build.BuildNumber)'
161+
releaseNotesSource: inline
162+
releaseNotes: 'The release process is automated.'
163+
assets: '$(System.DefaultWorkingDirectory)/github/*.*'
164+

‎deploy/docker-compose/docker-compose.override.dev.yaml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
version: '3'
2+
services:
3+
sftp:
4+
image: "emberstack/sftp:dev"
5+
build:
6+
context: ../../src
7+
dockerfile: Dockerfile
8+
ports:
9+
- "2222:22"
10+
volumes:
11+
- ../samples/sample.dev.sftp.json:/sftp/config/sftp.json:ro
12+
- ../samples/.ssh/id_demo_rsa.pub:/home/demo/.ssh/keys/id_rsa.pub:ro
13+
- ../samples/.ssh/id_demo2_rsa.pub:/home/demo2/.ssh/keys/id_rsa.pub:ro

‎deploy/docker-compose/docker-compose.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
version: '3'
2+
services:
3+
sftp:
4+
image: "emberstack/sftp"
5+
ports:
6+
- "22:22"
7+
volumes:
8+
- ../samples/sample.sftp.json:/sftp/config/sftp.json:ro

‎deploy/helm/sftp/.helmignore

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*~
18+
# Various IDEs
19+
.project
20+
.idea/
21+
*.tmproj
22+
.vscode/

0 commit comments

Comments
 (0)