Skip to content

Commit d172e70

Browse files
princeparmarPradip Parmarcnlangzivincenzopalazzo
authored
added feed and live command with upload. (#162)
* added feed and live command with upload. Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * changed readme file for feed and live commands Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * read me updated for live and feed command Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * Update README.md Co-authored-by: Lz <imlangzi@qq.com> * Update README.md Co-authored-by: Lz <imlangzi@qq.com> * Update README.md Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * readme updates from suggestion -- removed sync and live flag from uploda. from zbox code and readme both. Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * image updated for sync Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * updated flag description in code. Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * image url update Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * Update README.md * Update cmd/upload.go Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> * Update cmd/upload.go Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> * Update cmd/upload.go Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> * build fix Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * added feed and live command with upload. Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * changed readme file for feed and live commands Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * read me updated for live and feed command Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * Update README.md Co-authored-by: Lz <imlangzi@qq.com> * Update README.md Co-authored-by: Lz <imlangzi@qq.com> * Update README.md Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * Update cmd/upload.go Co-authored-by: Lz <imlangzi@qq.com> * readme updates from suggestion -- removed sync and live flag from uploda. from zbox code and readme both. Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * image updated for sync Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * updated flag description in code. Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * image url update Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> * Update README.md * Update cmd/upload.go Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> * Update cmd/upload.go Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> * Update cmd/upload.go Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com> * build fix Signed-off-by: Pradip Parmar <pradip@alfa-creator.com> Co-authored-by: Pradip Parmar <pradip@alfa-creator.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
1 parent eb3ff92 commit d172e70

File tree

2 files changed

+247
-39
lines changed

2 files changed

+247
-39
lines changed

‎README.md

Lines changed: 56 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ zbox is a command line interface (CLI) tool to understand the capabilities of 0C
3030
- [Update blobber settings](#update-blobber-settings)
3131
- [Uploading and Managing Files](#uploading-and-managing-files)
3232
- [Upload](#upload)
33+
- [Stream](#stream)
34+
- [Feed](#feed)
3335
- [Download](#download)
3436
- [Update](#update)
3537
- [Delete](#delete)
@@ -673,12 +675,6 @@ The user must be the owner of the allocation.You can request the file be encrypt
673675
| remotepath | yes | remote path to upload file to, use to access file later | | string |
674676
| thumbnailpath | no | local path of thumbnaSil | | file path |
675677
| chunksize | no | chunk size | 65536 | int |
676-
| delay | no | set segment duration to seconds. only works with --live and --sync.| 5 | int |
677-
| sync | no | enable SyncUpload from remote live feed. disabled it by default. | false | boolean |
678-
| feed | no | set remote live feed to url. only works with --sync. | false | url |
679-
| downloader-args | no | pass args to youtube-dl to download video. default is \"-q -f best\". only works with --sync.| -q -f best | [youtube-dl](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options)|
680-
| ffmpeg-args | no | pass args to ffmpeg to build segments. only works with --sync. | -loglevel warning | [ffmpeg](https://www.ffmpeg.org/ffmpeg.html) |
681-
| live | no | enable LiveUpload from local devices. disabled by default. | false | boolean |
682678

683679

684680
<details>
@@ -720,9 +716,60 @@ Response:
720716
Status completed callback. Type = application/octet-stream. Name = sensitivedata.txt
721717
```
722718

723-
**Download segment files from remote live feed, re-encode and upload**
719+
## Stream
720+
721+
Use `stream` to capture video and audio streaming form local devices, and upload
722+
723+
The user must be the owner of the allocation.You can request the file be encrypted before upload, and can send thumbnails with the file.
724+
725+
| Parameter | Required | Description | Default | Valid values |
726+
|-------------------------|----------|------------------------------------------------------|---------|-----------------------------------------|
727+
| allocation | yes | allocation id, sender must be allocation owner | | string |
728+
| commit | no | save metadata to blockchain | false | boolean |
729+
| encrypt | no | encrypt file before upload | false | boolean |
730+
| localpath | yes | local path of segment files to download, generate and upload | | file path |
731+
| remotepath | yes | remote path to upload file to, use to access file later | | string |
732+
| thumbnailpath | no | local path of thumbnaSil | | file path |
733+
| chunksize | no | chunk size | 65536 | int |
734+
| delay | no | set segment duration to seconds. | 5 | int |
735+
736+
<details>
737+
<summary>stream</summary>
738+
739+
![image](https://github.com/0chain/blobber/wiki/uml/usecase/live_upload_live.png)
740+
741+
</details>
742+
743+
## Feed
744+
745+
Use `feed` command to automatically download segment files from remote live feed with `--downloader-args "-q -f best"`
746+
- encode them into new segment files with `--delay` and `--ffmpeg-args`, and upload.
747+
- please use `youtube-dl -F https://www.youtube.com/watch?v=pC5mGB5enkw` to list formats of video (see below).
748+
749+
The user must be the owner of the allocation.You can request the file be encrypted before upload, and can send thumbnails with the file.
750+
751+
| Parameter | Required | Description | Default | Valid values |
752+
|-------------------------|----------|------------------------------------------------------|---------|-----------------------------------------|
753+
| allocation | yes | allocation id, sender must be allocation owner | | string |
754+
| commit | no | save metadata to blockchain | false | boolean |
755+
| encrypt | no | encrypt file before upload | false | boolean |
756+
| localpath | yes | local path of segment files to download, generate and upload | | file path |
757+
| remotepath | yes | remote path to upload file to, use to access file later | | string |
758+
| thumbnailpath | no | local path of thumbnaSil | | file path |
759+
| chunksize | no | chunk size | 65536 | int |
760+
| delay | no | set segment duration to seconds. | 5 | int |
761+
| feed | no | set remote live feed to url. | false | url |
762+
| downloader-args | no | pass args to youtube-dl to download video. default is \"-q -f best\". | -q -f best | [youtube-dl](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options)|
763+
| ffmpeg-args | no | pass args to ffmpeg to build segments. | -loglevel warning | [ffmpeg](https://www.ffmpeg.org/ffmpeg.html) |
764+
765+
766+
<details>
767+
<summary>feed</summary>
768+
769+
![image](https://github.com/0chain/blobber/wiki/uml/usecase/live_upload_sync.png)
770+
771+
</details>
724772

725-
Use `upload --sync` command to automatically download segment files from remove live feed with `--downloader-args "-f 22"`, encode them into new segment files with `--delay` and `--ffmpeg-args`, and upload. please use `youtube-dl -F https://www.youtube.com/watch?v=pC5mGB5enkw` to list formats of video (see below).
726773

727774
```
728775
[youtube] pC5mGB5enkw: Downloading webpage
@@ -753,18 +800,9 @@ format code extension resolution note
753800
`--downloader-args "-f 22"` dowloads video with `22 mp4 1280x676 720p 2117k , avc1.64001F, 30fps, mp4a.40.2 (44100Hz) (best)`
754801

755802
```
756-
./zbox upload --sync --localpath <absolute path to file>/tvshow.m3u8 --remotepath /videos/tvsho --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac --delay 10 --chunksize 655360 --downloader-args "-f 22" --feed https://www.youtube.com/watch?v=pC5mGB5enkw
757-
758-
```
759-
760-
**Capture streaming from local devices, encode with ffmpeg, and upload**
761-
762-
Use `upload --live` to capture video and audio streaming form local devices, and upload
803+
./zbox feed --localpath <absolute path to file>/tvshow.m3u8 --remotepath /videos/tvsho --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac --delay 10 --chunksize 655360 --downloader-args "-f 22" --feed https://www.youtube.com/watch?v=pC5mGB5enkw
763804
764805
```
765-
./zbox upload --live --localpath <absolute path to file>/streaming.m3u8 --remotepath /videos/streaming --allocation d0939e912851959637257573b08c748474f0dd0ebbc8e191e4f6ad69e4fdc7ac --delay 10 --chunksize 655360
766-
```
767-
768806

769807
## Download
770808

‎cmd/upload.go

Lines changed: 191 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -113,20 +113,160 @@ var uploadCmd = &cobra.Command{
113113
attrs.WhoPaysForReads = wp // set given value
114114
}
115115

116-
live, _ := cmd.Flags().GetBool("live")
117-
sync, _ := cmd.Flags().GetBool("sync")
118116
chunkSize, _ := cmd.Flags().GetInt("chunksize")
119117

120-
if live {
121-
// capture video and audio from local default camera and micrlphone, and upload it to zcn
122-
err = startLiveUpload(cmd, allocationObj, localpath, remotepath, encrypt, chunkSize, attrs)
123-
} else if sync {
124-
// download video from remote live feed(eg youtube), and sync it to zcn
125-
err = startSyncUpload(cmd, allocationObj, localpath, remotepath, encrypt, chunkSize, attrs)
126-
} else {
127-
err = startChunkedUpload(cmd, allocationObj, localpath, thumbnailpath, remotepath, encrypt, chunkSize, attrs, statusBar, false)
118+
if err := startChunkedUpload(cmd, allocationObj, localpath, thumbnailpath, remotepath, encrypt, chunkSize, attrs, statusBar, false); err != nil {
119+
PrintError("Upload failed.", err)
120+
os.Exit(1)
121+
}
122+
wg.Wait()
123+
if !statusBar.success {
124+
os.Exit(1)
125+
}
126+
127+
if commit {
128+
remotepath = zboxutil.GetFullRemotePath(localpath, remotepath)
129+
statusBar.wg.Add(1)
130+
commitMetaTxn(remotepath, "Upload", "", "", allocationObj, nil, statusBar)
131+
statusBar.wg.Wait()
132+
}
133+
},
134+
}
135+
136+
// feedCmd represents upload command with --sync flag
137+
var feedCmd = &cobra.Command{
138+
Use: "feed",
139+
Short: "download segment files from remote live feed, and upload",
140+
Long: "download segment files from remote live feed, and upload",
141+
Args: cobra.MinimumNArgs(0),
142+
Run: func(cmd *cobra.Command, args []string) {
143+
fflags := cmd.Flags() // fflags is a *flag.FlagSet
144+
if !fflags.Changed("allocation") { // check if the flag "path" is set
145+
PrintError("Error: allocation flag is missing") // If not, we'll let the user know
146+
os.Exit(1) // and return
147+
}
148+
if !fflags.Changed("remotepath") {
149+
PrintError("Error: remotepath flag is missing")
150+
os.Exit(1)
151+
}
152+
153+
if !fflags.Changed("localpath") {
154+
PrintError("Error: localpath flag is missing")
155+
os.Exit(1)
156+
}
157+
158+
allocationID := cmd.Flag("allocation").Value.String()
159+
allocationObj, err := sdk.GetAllocation(allocationID)
160+
if err != nil {
161+
PrintError("Error fetching the allocation.", err)
162+
os.Exit(1)
163+
}
164+
remotepath := cmd.Flag("remotepath").Value.String()
165+
localpath := cmd.Flag("localpath").Value.String()
166+
encrypt, _ := cmd.Flags().GetBool("encrypt")
167+
commit, _ := cmd.Flags().GetBool("commit")
168+
169+
wg := &sync.WaitGroup{}
170+
statusBar := &StatusBar{wg: wg}
171+
wg.Add(1)
172+
if strings.HasPrefix(remotepath, "/Encrypted") {
173+
encrypt = true
174+
}
175+
var attrs fileref.Attributes
176+
if fflags.Changed("attr-who-pays-for-reads") {
177+
var (
178+
wp common.WhoPays
179+
wps string
180+
)
181+
if wps, err = fflags.GetString("attr-who-pays-for-reads"); err != nil {
182+
log.Fatalf("getting 'attr-who-pays-for-reads' flag: %v", err)
183+
}
184+
if err = wp.Parse(wps); err != nil {
185+
log.Fatal(err)
186+
}
187+
attrs.WhoPaysForReads = wp // set given value
188+
}
189+
190+
chunkSize, _ := cmd.Flags().GetInt("chunksize")
191+
192+
// download video from remote live feed(eg youtube), and sync it to zcn
193+
err = startSyncUpload(cmd, allocationObj, localpath, remotepath, encrypt, chunkSize, attrs)
194+
195+
if err != nil {
196+
PrintError("Upload failed.", err)
197+
os.Exit(1)
198+
}
199+
wg.Wait()
200+
if !statusBar.success {
201+
os.Exit(1)
202+
}
203+
204+
if commit {
205+
remotepath = zboxutil.GetFullRemotePath(localpath, remotepath)
206+
statusBar.wg.Add(1)
207+
commitMetaTxn(remotepath, "Upload", "", "", allocationObj, nil, statusBar)
208+
statusBar.wg.Wait()
209+
}
210+
},
211+
}
212+
213+
// streamCmd represents upload command with --live flag
214+
var streamCmd = &cobra.Command{
215+
Use: "stream",
216+
Short: "capture video and audio streaming form local devices, and upload",
217+
Long: "capture video and audio streaming form local devices, and upload",
218+
Args: cobra.MinimumNArgs(0),
219+
Run: func(cmd *cobra.Command, args []string) {
220+
fflags := cmd.Flags() // fflags is a *flag.FlagSet
221+
if !fflags.Changed("allocation") { // check if the flag "path" is set
222+
PrintError("Error: allocation flag is missing") // If not, we'll let the user know
223+
os.Exit(1) // and return
224+
}
225+
if !fflags.Changed("remotepath") {
226+
PrintError("Error: remotepath flag is missing")
227+
os.Exit(1)
228+
}
229+
230+
if !fflags.Changed("localpath") {
231+
PrintError("Error: localpath flag is missing")
232+
os.Exit(1)
128233
}
129234

235+
allocationID := cmd.Flag("allocation").Value.String()
236+
allocationObj, err := sdk.GetAllocation(allocationID)
237+
if err != nil {
238+
PrintError("Error fetching the allocation.", err)
239+
os.Exit(1)
240+
}
241+
remotepath := cmd.Flag("remotepath").Value.String()
242+
localpath := cmd.Flag("localpath").Value.String()
243+
encrypt, _ := cmd.Flags().GetBool("encrypt")
244+
commit, _ := cmd.Flags().GetBool("commit")
245+
246+
wg := &sync.WaitGroup{}
247+
statusBar := &StatusBar{wg: wg}
248+
wg.Add(1)
249+
if strings.HasPrefix(remotepath, "/Encrypted") {
250+
encrypt = true
251+
}
252+
var attrs fileref.Attributes
253+
if fflags.Changed("attr-who-pays-for-reads") {
254+
var wp common.WhoPays
255+
var wps string
256+
if wps, err = fflags.GetString("attr-who-pays-for-reads"); err != nil {
257+
log.Fatalf("getting 'attr-who-pays-for-reads' flag: %v", err)
258+
}
259+
if err = wp.Parse(wps); err != nil {
260+
log.Fatal(err)
261+
}
262+
attrs.WhoPaysForReads = wp // set given value
263+
}
264+
265+
chunkSize, _ := cmd.Flags().GetInt("chunksize")
266+
267+
// capture video and audio from local default camera and micrlphone, and upload it to zcn
268+
err = startLiveUpload(cmd, allocationObj, localpath, remotepath, encrypt, chunkSize, attrs)
269+
130270
if err != nil {
131271
PrintError("Upload failed.", err)
132272
os.Exit(1)
@@ -310,17 +450,6 @@ func init() {
310450

311451
uploadCmd.Flags().Int("chunksize", sdk.CHUNK_SIZE, "chunk size")
312452

313-
uploadCmd.Flags().Int("delay", 5, "set segment duration to seconds. only works with --live and --sync. default duration is 5s.")
314-
315-
// SyncUpload
316-
uploadCmd.Flags().Bool("sync", false, "enable SyncUpload from remote live feed. disabled by default.")
317-
uploadCmd.Flags().String("feed", "", "set remote live feed to url. only works with --sync.")
318-
uploadCmd.Flags().String("downloader-args", "-q -f best", "pass args to youtube-dl to download video. default is \"-q\". only works with --sync.")
319-
uploadCmd.Flags().String("ffmpeg-args", "-loglevel warning", "pass args to ffmpeg to build segments. default is \"-loglevel warning\". only works with --sync.")
320-
321-
// LiveUpload
322-
uploadCmd.Flags().Bool("live", false, "enable LiveUpload from local devices. disabled by default.")
323-
324453
uploadCmd.MarkFlagRequired("allocation")
325454
uploadCmd.MarkFlagRequired("remotepath")
326455
uploadCmd.MarkFlagRequired("localpath")
@@ -330,4 +459,45 @@ func init() {
330459
createDirCmd.MarkFlagRequired("allocation")
331460
createDirCmd.MarkFlagRequired("dirname")
332461

462+
// feed command
463+
rootCmd.AddCommand(feedCmd)
464+
feedCmd.PersistentFlags().String("allocation", "", "Allocation ID")
465+
feedCmd.PersistentFlags().String("remotepath", "", "Remote path to upload")
466+
feedCmd.PersistentFlags().String("localpath", "", "Local path of file to upload")
467+
feedCmd.PersistentFlags().String("thumbnailpath", "", "Local thumbnail path of file to upload")
468+
feedCmd.PersistentFlags().String("attr-who-pays-for-reads", "owner", "Who pays for reads: owner or 3rd_party")
469+
feedCmd.Flags().Bool("encrypt", false, "pass this option to encrypt and upload the file")
470+
feedCmd.Flags().Bool("commit", false, "pass this option to commit the metadata transaction")
471+
472+
feedCmd.Flags().Int("chunksize", sdk.CHUNK_SIZE, "chunk size")
473+
474+
feedCmd.Flags().Int("delay", 5, "set segment duration to seconds.")
475+
476+
// SyncUpload
477+
feedCmd.Flags().String("feed", "", "set remote live feed to url.")
478+
feedCmd.Flags().String("downloader-args", "-q -f best", "pass args to youtube-dl to download video. default is \"-q\".")
479+
feedCmd.Flags().String("ffmpeg-args", "-loglevel warning", "pass args to ffmpeg to build segments. default is \"-loglevel warning\".")
480+
481+
feedCmd.MarkFlagRequired("allocation")
482+
feedCmd.MarkFlagRequired("remotepath")
483+
feedCmd.MarkFlagRequired("localpath")
484+
485+
// stream Command
486+
rootCmd.AddCommand(streamCmd)
487+
streamCmd.PersistentFlags().String("allocation", "", "Allocation ID")
488+
streamCmd.PersistentFlags().String("remotepath", "", "Remote path to upload")
489+
streamCmd.PersistentFlags().String("localpath", "", "Local path of file to upload")
490+
streamCmd.PersistentFlags().String("thumbnailpath", "", "Local thumbnail path of file to upload")
491+
streamCmd.PersistentFlags().String("attr-who-pays-for-reads", "owner", "Who pays for reads: owner or 3rd_party")
492+
streamCmd.Flags().Bool("encrypt", false, "pass this option to encrypt and upload the file")
493+
streamCmd.Flags().Bool("commit", false, "pass this option to commit the metadata transaction")
494+
495+
streamCmd.Flags().Int("chunksize", sdk.CHUNK_SIZE, "chunk size")
496+
497+
streamCmd.Flags().Int("delay", 5, "set segment duration to seconds.")
498+
499+
streamCmd.MarkFlagRequired("allocation")
500+
streamCmd.MarkFlagRequired("remotepath")
501+
streamCmd.MarkFlagRequired("localpath")
502+
333503
}

0 commit comments

Comments
 (0)