I have a file name called temp.csv in my script one of the intermediate step is echo some info > final.csv | cat temp.csv >> final.csvat times the file final.csv is created without data which is in temp.csv (when it is running through scheduler). And then, when I rerun the job, then the final.csv is created as I expected.Why it is happening like this (what exactly happening in the command echo some info > final.csv | cat temp.csv >> final.csv)?If I replace the command in the following way:echo some info > final.csv ; cat temp.csv >> final.csv will this modification be helpful?
Combination of echo, pipe and cat
shell;io redirection;pipe
You are piping the output from echo some info > final.csv into cat temp.csv >> final.csv, so these two commands (echo and cat) run in parallel.Because of that, what ends up in the final.csv depends on what program gets scheduled. What you want to do is replace that | with a ;. And then the echo command will run until finished, and only then cat is started.