Overview of what a “pipeline” is
In the Murex docs you’ll often see the term “pipeline”. This refers to any commands sequenced together.
A pipeline can be joined via any pipe token (eg |, ->, =>, ?). But, for the sake of documentation, a pipeline might even be a solitary command.
Typical Murex pipeline:
open example.json -> [[ /node/0 ]]Example of a single command pipeline:
topPipeline you might see in Bash / Zsh (this is also valid in Murex):
cat names.txt | sort | uniqPipeline filtering out a specific error from example-cmd
example-cmd ? grep "File not found"A pipeline isn’t a Murex specific construct but rather something inherited from Unix. Where Murex differs is that it can support sending typed information to compatible functions (unlike standard Unix pipes which are dumb-byte streams).
Wikipedia has a page on Pipeline (Unix):
In Unix-like computer operating systems, a pipeline is a mechanism for inter-process communication using message passing. A pipeline is a set of processes chained together by their standard streams, so that the output text of each process (stdout) is passed directly as input (stdin) to the next one. The second process is started as the first process is still executing, and they are executed concurrently. The concept of pipelines was championed by Douglas McIlroy at Unix’s ancestral home of Bell Labs, during the development of Unix, shaping its toolbox philosophy. It is named by analogy to a physical pipeline. A key feature of these pipelines is their “hiding of internals” (Ritchie & Thompson, 1974). This in turn allows for more clarity and simplicity in the system.
The drawback with pipes is that it assumes each command runs sequentially one after another and that everything fits neatly into the concept of “output” and “errors”. The moment you need to use background (bg) processes, do anything more specific with data streams (even if just ignore them entirely), or use more than one data stream, then this concept breaks down. This is where named pipes come to the rescue. Named pipes are out of scope for this specific document but you can read more on them in links the links below.
bg): Run processes in the background-> Arrow Pipe: Pipes stdout from the left hand command to stdin of the right hand command=> Generic Pipe: Pipes a reformatted stdout stream from the left hand command to stdin of the right hand command? stderr Pipe: Pipes stderr from the left hand command to stdin of the right hand command (DEPRECATED)| POSIX Pipe: Pipes stdout from the left hand command to stdin of the right hand commandThis document was generated from gen/user-guide/pipeline_doc.yaml.
This site's content is rebuilt automatically from murex's source code after each merge to the master branch. Downloadable murex binaries are also built with the website.
Last built on Sat Nov 23 00:50:15 UTC 2024 against commit 69c17da69c17da3bd9db98ca508f6a03a402f074ee24cec.
Current version is 6.4.0375 (develop) which has been verified against tests cases.