File tree Expand file tree Collapse file tree 1 file changed +11
-14
lines changed Expand file tree Collapse file tree 1 file changed +11
-14
lines changed Original file line number Diff line number Diff line change @@ -624,26 +624,23 @@ df = pd.read_json(src)
624
624
625
625
626
626
``` python
627
- dst = " hf://datasets/kszucs/pq/hermes-2.5-cdc.parquet"
628
- df.to_parquet(dst, use_content_defined_chunking = True )
627
+ >> > dst = " hf://datasets/kszucs/pq/hermes-2.5-cdc.parquet"
628
+ >> > df.to_parquet(dst, use_content_defined_chunking = True )
629
+ New Data Upload: 100 % | ███████████████████████████████████████████████| 799MB / 799MB , 197kB / s
630
+ Total Bytes: 799M
631
+ Total Transfer: 799M
629
632
```
630
633
631
- New Data Upload: 100%|███████████████████████████████████████████████| 799MB / 799MB, 197kB/s
632
- Total Bytes: 799M
633
- Total Transfer: 799M
634
-
635
-
636
634
637
635
``` python
638
- short_df = df[[len (c) < 10 for c in df.conversations]]
639
- short_dst = " hf://datasets/kszucs/pq/hermes-2.5-cdc-short.parquet"
640
- short_df.to_parquet(short_dst, use_content_defined_chunking = True )
636
+ >> > short_df = df[[len (c) < 10 for c in df.conversations]]
637
+ >> > short_dst = " hf://datasets/kszucs/pq/hermes-2.5-cdc-short.parquet"
638
+ >> > short_df.to_parquet(short_dst, use_content_defined_chunking = True )
639
+ New Data Upload: 100 % | ███████████████████████████████████████████████| 21. 9MB / 21. 9MB , 45. 4kB / s
640
+ Total Bytes: 801M
641
+ Total Transfer: 21. 9M
641
642
```
642
643
643
- New Data Upload: 100%|███████████████████████████████████████████████| 21.9MB / 21.9MB, 45.4kB/s
644
- Total Bytes: 801M
645
- Total Transfer: 21.9M
646
-
647
644
648
645
649
646
``` python
You can’t perform that action at this time.
0 commit comments