Practical Data Science Using the Shell

For data analysis tasks, typically we load the data into a dedicated tool, i.e. the statistic program R,
mathematica, a relational database, or some other specialized tools to perform our analysis. However, there is often another option that can be performed on almost any computer. The GNU core utils are available by default on many computers and provide a set of powerful tools for manipulating and transforming data and also for performing analyses such as aggregation, etc. In addition to being freely available, these tools have the advantage that they can be used immediately, without the data having to be transformed and loaded into the target system beforehand. The stream-based approach throughout means that even very large amounts of data can be processed without running out of main memory.