Geek asks and answers

How to split a file into 250 thousand columns vertically?

I need to divide the file into 250 thousand based on the size (preferred) or the number of columns into several (~ 5) pieces. I know a command splitfor breaking down rows, but I don’t know if there is any similar function for breaking down columns. The number of columns in my file is not equal, so the pieces cannot have an equal number of columns.

Input:

AA BB CC DD EE FF GG HH II JJ KK LL MM
NN OO PP QQ RR SS TT UU VV WW XX YY ZZ

Required Conclusion:

File1

AA BB CC DD        
NN OO PP QQ

File2

EE FF GG HH
RR SS TT UU

File3

II JJ KK LL MM
VV WW XX YY ZZ
+3
source share
3 answers

Using awk, you can tune n to the number you expect.

awk '{for (i=1;i<=NF;i++)
         printf (i%n==0||i==NF)?$i RS:$i FS > "File" int((i-1)/n+1) ".txt"
      }' n=5 file
+3
source

Use cut. This is part of GNU coreutils.

Assuming your input file columns are separated by a space:

cut -d " " -f1-4 /path/to/input/file > file1

cut -d " " -f5-8 /path/to/input/file > file2

...

. man cut.

+2

I would use awkfor this. Not sure if you want to have 5 columns per file, as you mentioned that you have 250k columns that would create 50K files, but here is what you need to start:

awk '{
  y=1
  for(i=1;i<NF;i++) { 
    if(i%5==0) {
      print $i > "text"y".txt"
      y+=1
      continue 
    }
  printf "%s ",$i >"text"y".txt"
  } 
print $NF > "text"y".txt"}' file

Test:

$ cat file
AA BB CC DD EE FF GG HH II JJ KK LL MM
NN OO PP QQ RR SS TT UU VV WW XX YY ZZ

$ awk '{
  y=1
  for(i=1;i<NF;i++) { 
    if(i%5==0) {
      print $i > "text"y".txt"
      y+=1
      continue 
    }
  printf "%s ",$i >"text"y".txt"
  } 
print $NF > "text"y".txt"}' file

$ head text*
==> text1.txt <==
AA BB CC DD EE
NN OO PP QQ RR

==> text2.txt <==
FF GG HH II JJ
SS TT UU VV WW

==> text3.txt <==
KK LL MM
XX YY ZZ
+2
source

More articles:


All Articles