mirrors/ctrl
mirrors/ctrl
1
0
Fork 0
mirror of https://github.com/postmannen/ctrl.git synced 2024-12-14 12:37:31 +00:00
Code Activity

more about messaging added to the book

Browse source
This commit is contained in:
postmannen 2024-11-18 11:27:32 +01:00
parent fc15c4dd63
commit 8ec3e8c21e
16 changed files with 599 additions and 24 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -107,7 +107,7 @@ ctrl supports both the use of flags with env variables. An .env file can also be
### Schema for the messages to send into ctrl via the API's
|Name | value | description|
|Field Name | Value Type | Description|
|-----|-------|------------|
|toNode | `string` | A single node to send a message to|
|toNodes | `string array` | A comma separated list of nodes to send a message to|

287
doc/book/core_messaging_message_fields.html Normal file
View file

@ -0,0 +1,287 @@
<!DOCTYPE HTML>
<html lang="en" class="light sidebar-visible" dir="ltr">
<head>
<!-- Book generated using mdBook -->
<meta charset="UTF-8">
<title>Message fields - ctrl</title>
<!-- Custom HTML head -->
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#ffffff">
<link rel="icon" href="favicon.svg">
<link rel="shortcut icon" href="favicon.png">
<link rel="stylesheet" href="css/variables.css">
<link rel="stylesheet" href="css/general.css">
<link rel="stylesheet" href="css/chrome.css">
<link rel="stylesheet" href="css/print.css" media="print">
<!-- Fonts -->
<link rel="stylesheet" href="FontAwesome/css/font-awesome.css">
<link rel="stylesheet" href="fonts/fonts.css">
<!-- Highlight.js Stylesheets -->
<link rel="stylesheet" href="highlight.css">
<link rel="stylesheet" href="tomorrow-night.css">
<link rel="stylesheet" href="ayu-highlight.css">
<!-- Custom theme stylesheets -->
<!-- Provide site root to javascript -->
<script>
var path_to_root = "";
var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "navy" : "light";
</script>
<!-- Start loading toc.js asap -->
<script src="toc.js"></script>
</head>
<body>
<div id="body-container">
<!-- Work around some values being stored in localStorage wrapped in quotes -->
<script>
try {
var theme = localStorage.getItem('mdbook-theme');
var sidebar = localStorage.getItem('mdbook-sidebar');
if (theme.startsWith('"') && theme.endsWith('"')) {
localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
}
if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
}
} catch (e) { }
</script>
<!-- Set the theme before any content is loaded, prevents flash -->
<script>
var theme;
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
if (theme === null || theme === undefined) { theme = default_theme; }
const html = document.documentElement;
html.classList.remove('light')
html.classList.add(theme);
html.classList.add("js");
</script>
<input type="checkbox" id="sidebar-toggle-anchor" class="hidden">
<!-- Hide / unhide sidebar before it is displayed -->
<script>
var sidebar = null;
var sidebar_toggle = document.getElementById("sidebar-toggle-anchor");
if (document.body.clientWidth >= 1080) {
try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
sidebar = sidebar || 'visible';
} else {
sidebar = 'hidden';
}
sidebar_toggle.checked = sidebar === 'visible';
html.classList.remove('sidebar-visible');
html.classList.add("sidebar-" + sidebar);
</script>
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
<!-- populated by js -->
<mdbook-sidebar-scrollbox class="sidebar-scrollbox"></mdbook-sidebar-scrollbox>
<noscript>
<iframe class="sidebar-iframe-outer" src="toc.html"></iframe>
</noscript>
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
<div class="sidebar-resize-indicator"></div>
</div>
</nav>
<div id="page-wrapper" class="page-wrapper">
<div class="page">
<div id="menu-bar-hover-placeholder"></div>
<div id="menu-bar" class="menu-bar sticky">
<div class="left-buttons">
<label id="sidebar-toggle" class="icon-button" for="sidebar-toggle-anchor" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
<i class="fa fa-bars"></i>
</label>
<button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
<i class="fa fa-paint-brush"></i>
</button>
<ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
<li role="none"><button role="menuitem" class="theme" id="light">Light</button></li>
<li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
<li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
<li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
<li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
</ul>
<button id="search-toggle" class="icon-button" type="button" title="Search. (Shortkey: s)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="S" aria-controls="searchbar">
<i class="fa fa-search"></i>
</button>
</div>
<h1 class="menu-title">ctrl</h1>
<div class="right-buttons">
<a href="print.html" title="Print this book" aria-label="Print this book">
<i id="print-button" class="fa fa-print"></i>
</a>
</div>
</div>
<div id="search-wrapper" class="hidden">
<form id="searchbar-outer" class="searchbar-outer">
<input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
</form>
<div id="searchresults-outer" class="searchresults-outer hidden">
<div id="searchresults-header" class="searchresults-header"></div>
<ul id="searchresults">
</ul>
</div>
</div>
<!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
<script>
document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
});
</script>
<div id="content" class="content">
<main>
<h1 id="message-fields"><a class="header" href="#message-fields">Message fields</a></h1>
<pre><code class="language-yaml">toNode : "some-node"
</code></pre>
<p>The node to send the message to.</p>
<pre><code class="language-yaml">toNodes : node1,node2
</code></pre>
<p>ToNodes to specify several hosts to send message to in the form of an slice/array. When used, the message will be split up into one message for each node specified, so the sending to each node will be handled individually.</p>
<pre><code class="language-yaml">data : data here in byte format
</code></pre>
<p>The actual data in the message. This is the field where we put the returned data in a reply message. The data field are of type []byte.</p>
<pre><code class="language-yaml">method : REQCliCommand
</code></pre>
<p>What request method type to use, like REQCliCommand, REQHttpGet..</p>
<pre><code class="language-yaml"> methodArgs :
- "bash"
- "-c"
- |
echo "this is a test"
echo "and some other test"
</code></pre>
<p>Additional arguments that might be needed when executing the method. Can be f.ex. an ip address if it is a tcp sender, or the actual shell command to execute in a cli.</p>
<pre><code class="language-yaml">replyMethod : REQToFile
</code></pre>
<p>ReplyMethod, is the method to use for the reply message. By default the reply method will be set to log to file, but you can override it setting your own here.</p>
<pre><code class="language-yaml"> methodArgs :
- "bash"
- "-c"
- |
echo "this is a test"
</code></pre>
<p>Additional arguments that might be needed when executing the reply method. Can be f.ex. an ip address if it is a tcp sender, or the shell command to execute in a cli session.
replyMethodArgs :</p>
<pre><code class="language-yaml">fromNode Node : "node2"
</code></pre>
<p>From what node the message originated. This field is automatically filled by ctrl when left blanc, so that when a message are sent from a node the user don't have to worry about getting eventual repply messages back.
Setting a</p>
<pre><code class="language-yaml">ACKTimeout: 10
</code></pre>
<p>ACKTimeout for waiting for an Ack message in seconds.</p>
<p>If the ACKTimeout value is set to 0 the message will become an No Ack message. With No Ack messages we will not wait for an Ack, nor will the receiver send an Ack, and we will never try to resend a message.</p>
<pre><code class="language-yaml">retryWait : 60
</code></pre>
<p>RetryWait specifies the time in seconds to wait between retries. This value is added to the ACKTimeout and to make the total time before retrying to sending a message. A usecase can be when you want a low ACKTimeout, but you want to add more time between the retries to avoid spamming the receiving node.</p>
<pre><code class="language-yaml">retries : 3
</code></pre>
<p>How many times we should try to resend the message if no ACK was received.</p>
<pre><code class="language-yaml">replyACKTimeout int `json:"replyACKTimeout" yaml:"replyACKTimeout"`
</code></pre>
<p>The ACK timeout used with reply messages. If not specified the value of <code>ACKTimeout</code> will be used.</p>
<pre><code class="language-yaml">replyRetries int `json:"replyRetries" yaml:"replyRetries"`
</code></pre>
<p>The number of retries for trying to resend a message for reply messages.</p>
<pre><code class="language-yaml">methodTimeout : 10
</code></pre>
<p>Timeout for how long a method should be allowed to run before it is timed out.</p>
<pre><code class="language-yaml">replyMethodTimeout : 10
</code></pre>
<p>Timeout for how long a method should be allowed to run before it is timed out, but for the reply message.</p>
<pre><code class="language-yaml">directory string `json:"directory" yaml:"directory"`
</code></pre>
<p>Specify the directory structure to use when saving the result data for a repply message.</p>
<ul>
<li>When the values are comma separated like <code>"syslog","metrics"</code> a syslog folder with a subfolder metrics will be created in the directory specified with startup env variable <code>SUBSCRIBER_DATA_FOLDER</code>.</li>
<li>Absolute paths can also be used if you want to save the result somewhere else on the system, like "/etc/myservice".</li>
</ul>
<pre><code class="language-yaml">fileName : myfile.conf
</code></pre>
<p>The fileName field are used together with the directory field mentioned earlier to create a full path for where to write the resulting data of a repply message.</p>
<pre><code class="language-yaml">schedule : [2,5]
</code></pre>
<p>Schedule are used for scheduling the method of messages to be executed several times. The schedule is defined as an array of two values, where the first value defines how often the schedule should trigger a run in seconds, and the second value is for how long the schedule are allowed to run in seconds total. <code>[2,5]</code> will trigger the method to be executed again every 2 seconds, but when a total time of 5 seconds have passed it will stop scheduling.</p>
</main>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="prev" href="core_messaging_overview.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<div style="clear: both"></div>
</nav>
</div>
</div>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="prev" href="core_messaging_overview.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
</nav>
</div>
<!-- Livereload script (if served using the cli tool) -->
<script>
const wsProtocol = location.protocol === 'https:' ? 'wss:' : 'ws:';
const wsAddress = wsProtocol + "//" + location.host + "/" + "__livereload";
const socket = new WebSocket(wsAddress);
socket.onmessage = function (event) {
if (event.data === "reload") {
socket.close();
location.reload();
}
};
window.onbeforeunload = function() {
socket.close();
}
</script>
<script>
window.playground_copyable = true;
</script>
<script src="elasticlunr.min.js"></script>
<script src="mark.min.js"></script>
<script src="searcher.js"></script>
<script src="clipboard.min.js"></script>
<script src="highlight.js"></script>
<script src="book.js"></script>
<!-- Custom JS scripts -->
</div>
</body>
</html>

35
doc/book/user_guide.html → doc/book/core_messaging_overview.html
View file

@ -3,7 +3,7 @@
<head>
<!-- Book generated using mdBook -->
<meta charset="UTF-8">
<title>User Guide - ctrl</title>
<title>Messaging - ctrl</title>
<!-- Custom HTML head -->
@ -152,16 +152,39 @@
<div id="content" class="content">
<main>
<h1 id="messaging"><a class="header" href="#messaging">Messaging</a></h1>
<p>The handling of all messages is done by spawning up a process for handling the message in it's own thread. This allows us to keep the state of each <strong>individual message level</strong> both in regards to ACK's, error handling, send retries, and reruns of methods for a message if the first run was not successful.</p>
<p>All messages processed by a publisher will be written to a log file after they are processed, with all the information needed to recreate the same message if needed, or it can be used for auditing.</p>
<p>All handling down to the process and message level are handled concurrently. So if there are problems handling one message sent to a node on a subject it will not affect the messages being sent to other nodes, or other messages sent on other subjects to the same host.</p>
<p>Message types of both <strong>ACK</strong> and <strong>NACK</strong>, so we can decide if we want or don't want an Acknowledge if a message was delivered succesfully.
Example: We probably want an <strong>ACK</strong> when sending some <strong>REQCLICommand</strong> to be executed, but we don't care for an acknowledge <strong>NACK</strong> when we send an <strong>REQHello</strong> event.
If a message are <strong>ACK</strong> or <strong>NACK</strong> type are defined by the value of the <strong>ACKTimeout</strong> for each individual message:</p>
<ul>
<li><strong>ACKTimeout</strong> set to 0 will make the message become a <strong>NACK</strong> message.</li>
<li><strong>ACKTimeout</strong> set to &gt;=1 will make the message become an <strong>ACK</strong> message.</li>
</ul>
<p>If ACK is expected, and not received, then the message will be retried the amount of time defined in the <strong>retries</strong> field of a message. If numer of retries are reached, and still no ACK received, the message will be discared, and a log message will be sent to the log server.</p>
<p>To make things easier, all timeouts used for messages can be set with env variables or flags at startup of ctrl. Locally specified timeout directly in a message will override the startup values, and can be used for more granularity when needed.</p>
<h2 id="example-of-message-flow"><a class="header" href="#example-of-message-flow">Example of message flow:</a></h2>
<style>
img {
background-color: #FFFFFF;
}
</style>
</head>
<body>
<p align="center"><img src="https://github.com/postmannen/ctrl/blob/main/doc/message-flow.svg?raw=true" /></p>
</body>
</main>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="prev" href="introduction.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<a rel="prev" href="user_guide_install_host.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="user_guide_install_docker.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="core_messaging_message_fields.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
@ -171,11 +194,11 @@
</div>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="prev" href="introduction.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<a rel="prev" href="user_guide_install_host.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="user_guide_install_docker.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="core_messaging_message_fields.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>

4
doc/book/index.html
View file

@ -172,7 +172,7 @@ sender.</p>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="next prefetch" href="user_guide.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="user_guide_install_docker.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
@ -183,7 +183,7 @@ sender.</p>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="next prefetch" href="user_guide.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="user_guide_install_docker.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>

4
doc/book/introduction.html
View file

@ -172,7 +172,7 @@ sender.</p>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="next prefetch" href="user_guide.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="user_guide_install_docker.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
@ -183,7 +183,7 @@ sender.</p>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="next prefetch" href="user_guide.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="user_guide_install_docker.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>

99
doc/book/print.html
View file

@ -167,7 +167,7 @@
<p>Ctrl is a system control tool that uses NATS as its messaging architecture. It allows you to send commands using request methods which are then executed on servers. If a receiving node is down, messages are retried based on the criteria set in their body. The results of these methods are delivered back to the
sender.</p>
<p>Ctrl is designed for concurrent processing and can handle multiple messages independently, even if some processes are slow or fail. It's compatible with various host OSs and systems including cloud containers, Raspberry Pi, and others with an installed operating system. Ctrl supports most major architectures such as x86, amd64, arm64, ppc64, and can run on operating systems like Linux, OSX, Windows.</p>
<div style="break-before: page; page-break-before: always;"></div><div style="break-before: page; page-break-before: always;"></div><h1 id="install-with-docker"><a class="header" href="#install-with-docker">Install with docker</a></h1>
<div style="break-before: page; page-break-before: always;"></div><h1 id="install-with-docker"><a class="header" href="#install-with-docker">Install with docker</a></h1>
<p>Start up a local nats message broker</p>
<pre><code class="language-bash">docker run -p 4444:4444 nats -p 4444
</code></pre>
@ -213,7 +213,8 @@ EOF
cp msg.yaml readfolder
</code></pre>
<p>With the above message we send to ourselves since we only got 1 node running. To start up more nodes repeat the above steps, replace.</p>
<p>With the above message we send to ourselves since we only got 1 node running. To start up more nodes repeat the above steps, replace the node name under <code>toNodes</code> with new names for new nodes.
NB: If more nodes share the same name the requests will be loadbalanced between them round robin.</p>
<div style="break-before: page; page-break-before: always;"></div><h1 id="install-on-a-host"><a class="header" href="#install-on-a-host">Install on a host</a></h1>
<p>Start up a local nats message broker if you don't already have one.</p>
<pre><code class="language-bash">docker run -p 4444:4444 nats -p 4444
@ -293,6 +294,100 @@ EOF
systemctl enable $progName.service &amp;&amp;
systemctl start $progName.service
</code></pre>
<div style="break-before: page; page-break-before: always;"></div><h1 id="messaging"><a class="header" href="#messaging">Messaging</a></h1>
<p>The handling of all messages is done by spawning up a process for handling the message in it's own thread. This allows us to keep the state of each <strong>individual message level</strong> both in regards to ACK's, error handling, send retries, and reruns of methods for a message if the first run was not successful.</p>
<p>All messages processed by a publisher will be written to a log file after they are processed, with all the information needed to recreate the same message if needed, or it can be used for auditing.</p>
<p>All handling down to the process and message level are handled concurrently. So if there are problems handling one message sent to a node on a subject it will not affect the messages being sent to other nodes, or other messages sent on other subjects to the same host.</p>
<p>Message types of both <strong>ACK</strong> and <strong>NACK</strong>, so we can decide if we want or don't want an Acknowledge if a message was delivered succesfully.
Example: We probably want an <strong>ACK</strong> when sending some <strong>REQCLICommand</strong> to be executed, but we don't care for an acknowledge <strong>NACK</strong> when we send an <strong>REQHello</strong> event.
If a message are <strong>ACK</strong> or <strong>NACK</strong> type are defined by the value of the <strong>ACKTimeout</strong> for each individual message:</p>
<ul>
<li><strong>ACKTimeout</strong> set to 0 will make the message become a <strong>NACK</strong> message.</li>
<li><strong>ACKTimeout</strong> set to &gt;=1 will make the message become an <strong>ACK</strong> message.</li>
</ul>
<p>If ACK is expected, and not received, then the message will be retried the amount of time defined in the <strong>retries</strong> field of a message. If numer of retries are reached, and still no ACK received, the message will be discared, and a log message will be sent to the log server.</p>
<p>To make things easier, all timeouts used for messages can be set with env variables or flags at startup of ctrl. Locally specified timeout directly in a message will override the startup values, and can be used for more granularity when needed.</p>
<h2 id="example-of-message-flow"><a class="header" href="#example-of-message-flow">Example of message flow:</a></h2>
<style>
img {
background-color: #FFFFFF;
}
</style>
</head>
<body>
<p align="center"><img src="https://github.com/postmannen/ctrl/blob/main/doc/message-flow.svg?raw=true" /></p>
</body>
<div style="break-before: page; page-break-before: always;"></div><h1 id="message-fields"><a class="header" href="#message-fields">Message fields</a></h1>
<pre><code class="language-yaml">toNode : "some-node"
</code></pre>
<p>The node to send the message to.</p>
<pre><code class="language-yaml">toNodes : node1,node2
</code></pre>
<p>ToNodes to specify several hosts to send message to in the form of an slice/array. When used, the message will be split up into one message for each node specified, so the sending to each node will be handled individually.</p>
<pre><code class="language-yaml">data : data here in byte format
</code></pre>
<p>The actual data in the message. This is the field where we put the returned data in a reply message. The data field are of type []byte.</p>
<pre><code class="language-yaml">method : REQCliCommand
</code></pre>
<p>What request method type to use, like REQCliCommand, REQHttpGet..</p>
<pre><code class="language-yaml"> methodArgs :
- "bash"
- "-c"
- |
echo "this is a test"
echo "and some other test"
</code></pre>
<p>Additional arguments that might be needed when executing the method. Can be f.ex. an ip address if it is a tcp sender, or the actual shell command to execute in a cli.</p>
<pre><code class="language-yaml">replyMethod : REQToFile
</code></pre>
<p>ReplyMethod, is the method to use for the reply message. By default the reply method will be set to log to file, but you can override it setting your own here.</p>
<pre><code class="language-yaml"> methodArgs :
- "bash"
- "-c"
- |
echo "this is a test"
</code></pre>
<p>Additional arguments that might be needed when executing the reply method. Can be f.ex. an ip address if it is a tcp sender, or the shell command to execute in a cli session.
replyMethodArgs :</p>
<pre><code class="language-yaml">fromNode Node : "node2"
</code></pre>
<p>From what node the message originated. This field is automatically filled by ctrl when left blanc, so that when a message are sent from a node the user don't have to worry about getting eventual repply messages back.
Setting a</p>
<pre><code class="language-yaml">ACKTimeout: 10
</code></pre>
<p>ACKTimeout for waiting for an Ack message in seconds.</p>
<p>If the ACKTimeout value is set to 0 the message will become an No Ack message. With No Ack messages we will not wait for an Ack, nor will the receiver send an Ack, and we will never try to resend a message.</p>
<pre><code class="language-yaml">retryWait : 60
</code></pre>
<p>RetryWait specifies the time in seconds to wait between retries. This value is added to the ACKTimeout and to make the total time before retrying to sending a message. A usecase can be when you want a low ACKTimeout, but you want to add more time between the retries to avoid spamming the receiving node.</p>
<pre><code class="language-yaml">retries : 3
</code></pre>
<p>How many times we should try to resend the message if no ACK was received.</p>
<pre><code class="language-yaml">replyACKTimeout int `json:"replyACKTimeout" yaml:"replyACKTimeout"`
</code></pre>
<p>The ACK timeout used with reply messages. If not specified the value of <code>ACKTimeout</code> will be used.</p>
<pre><code class="language-yaml">replyRetries int `json:"replyRetries" yaml:"replyRetries"`
</code></pre>
<p>The number of retries for trying to resend a message for reply messages.</p>
<pre><code class="language-yaml">methodTimeout : 10
</code></pre>
<p>Timeout for how long a method should be allowed to run before it is timed out.</p>
<pre><code class="language-yaml">replyMethodTimeout : 10
</code></pre>
<p>Timeout for how long a method should be allowed to run before it is timed out, but for the reply message.</p>
<pre><code class="language-yaml">directory string `json:"directory" yaml:"directory"`
</code></pre>
<p>Specify the directory structure to use when saving the result data for a repply message.</p>
<ul>
<li>When the values are comma separated like <code>"syslog","metrics"</code> a syslog folder with a subfolder metrics will be created in the directory specified with startup env variable <code>SUBSCRIBER_DATA_FOLDER</code>.</li>
<li>Absolute paths can also be used if you want to save the result somewhere else on the system, like "/etc/myservice".</li>
</ul>
<pre><code class="language-yaml">fileName : myfile.conf
</code></pre>
<p>The fileName field are used together with the directory field mentioned earlier to create a full path for where to write the resulting data of a repply message.</p>
<pre><code class="language-yaml">schedule : [2,5]
</code></pre>
<p>Schedule are used for scheduling the method of messages to be executed several times. The schedule is defined as an array of two values, where the first value defines how often the schedule should trigger a run in seconds, and the second value is for how long the schedule are allowed to run in seconds total. <code>[2,5]</code> will trigger the method to be executed again every 2 seconds, but when a total time of 5 seconds have passed it will stop scheduling.</p>
</main>

2
doc/book/searchindex.js
View file

File diff suppressed because one or more lines are too long

2
doc/book/searchindex.json
View file

File diff suppressed because one or more lines are too long

2
doc/book/toc.html
View file

@ -27,6 +27,6 @@
<!-- Custom theme stylesheets -->
</head>
<body class="sidebar-iframe-inner">
<ol class="chapter"><li class="chapter-item expanded "><a href="introduction.html" target="_parent"><strong aria-hidden="true">1.</strong> introduction</a></li><li class="chapter-item expanded "><a href="user_guide.html" target="_parent"><strong aria-hidden="true">2.</strong> User Guide</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="user_guide_install_docker.html" target="_parent"><strong aria-hidden="true">2.1.</strong> Install docker</a></li><li class="chapter-item expanded "><a href="user_guide_install_host.html" target="_parent"><strong aria-hidden="true">2.2.</strong> Install host</a></li></ol></li></ol>
<ol class="chapter"><li class="chapter-item expanded "><a href="introduction.html" target="_parent"><strong aria-hidden="true">1.</strong> introduction</a></li><li class="chapter-item expanded affix "><li class="part-title">User Guide</li><li class="chapter-item expanded "><a href="user_guide_install_docker.html" target="_parent"><strong aria-hidden="true">2.</strong> Install docker</a></li><li class="chapter-item expanded "><a href="user_guide_install_host.html" target="_parent"><strong aria-hidden="true">3.</strong> Install host</a></li><li class="chapter-item expanded affix "><li class="part-title">Core ctrl</li><li class="chapter-item expanded "><a href="core_messaging_overview.html" target="_parent"><strong aria-hidden="true">4.</strong> Messaging</a></li><li class="chapter-item expanded "><a href="core_messaging_message_fields.html" target="_parent"><strong aria-hidden="true">5.</strong> Message fields</a></li></ol>
</body>
</html>

2
doc/book/toc.js
View file

@ -8,7 +8,7 @@ class MDBookSidebarScrollbox extends HTMLElement {
super();
}
connectedCallback() {
this.innerHTML = '<ol class="chapter"><li class="chapter-item expanded "><a href="introduction.html"><strong aria-hidden="true">1.</strong> introduction</a></li><li class="chapter-item expanded "><a href="user_guide.html"><strong aria-hidden="true">2.</strong> User Guide</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="user_guide_install_docker.html"><strong aria-hidden="true">2.1.</strong> Install docker</a></li><li class="chapter-item expanded "><a href="user_guide_install_host.html"><strong aria-hidden="true">2.2.</strong> Install host</a></li></ol></li></ol>';
this.innerHTML = '<ol class="chapter"><li class="chapter-item expanded "><a href="introduction.html"><strong aria-hidden="true">1.</strong> introduction</a></li><li class="chapter-item expanded affix "><li class="part-title">User Guide</li><li class="chapter-item expanded "><a href="user_guide_install_docker.html"><strong aria-hidden="true">2.</strong> Install docker</a></li><li class="chapter-item expanded "><a href="user_guide_install_host.html"><strong aria-hidden="true">3.</strong> Install host</a></li><li class="chapter-item expanded affix "><li class="part-title">Core ctrl</li><li class="chapter-item expanded "><a href="core_messaging_overview.html"><strong aria-hidden="true">4.</strong> Messaging</a></li><li class="chapter-item expanded "><a href="core_messaging_message_fields.html"><strong aria-hidden="true">5.</strong> Message fields</a></li></ol>';
// Set the current, active page, and reveal it if it's hidden
let current_page = document.location.href.toString();
if (current_page.endsWith("/")) {

7
doc/book/user_guide_install_docker.html
View file

@ -198,13 +198,14 @@ EOF
cp msg.yaml readfolder
</code></pre>
<p>With the above message we send to ourselves since we only got 1 node running. To start up more nodes repeat the above steps, replace.</p>
<p>With the above message we send to ourselves since we only got 1 node running. To start up more nodes repeat the above steps, replace the node name under <code>toNodes</code> with new names for new nodes.
NB: If more nodes share the same name the requests will be loadbalanced between them round robin.</p>
</main>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="prev" href="user_guide.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<a rel="prev" href="introduction.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
@ -218,7 +219,7 @@ cp msg.yaml readfolder
</div>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="prev" href="user_guide.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<a rel="prev" href="introduction.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>

6
doc/book/user_guide_install_host.html
View file

@ -240,6 +240,9 @@ systemctl start $progName.service
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="core_messaging_overview.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
<div style="clear: both"></div>
</nav>
@ -251,6 +254,9 @@ systemctl start $progName.service
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="core_messaging_overview.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>
</div>

15
doc/src/SUMMARY.md
View file

@ -1,6 +1,13 @@
# Summary
# Introduction
- [introduction](./introduction.md)
- [User Guide](./user_guide.md)
- [Install docker](./user_guide_install_docker.md)
- [Install host](./user_guide_install_host.md)
# User Guide
- [Install docker](./user_guide_install_docker.md)
- [Install host](./user_guide_install_host.md)
# Core ctrl
- [Messaging](./core_messaging_overview.md)
- [Message fields](./core_messaging_message_fields.md)

1
doc/src/core_messaging.md Normal file
View file

@ -0,0 +1 @@
# Messaging

125
doc/src/core_messaging_message_fields.md Normal file
View file

@ -0,0 +1,125 @@
# Message fields
```yaml
toNode : "some-node"
```
The node to send the message to.
```yaml
toNodes : node1,node2
```
ToNodes to specify several hosts to send message to in the form of an slice/array. When used, the message will be split up into one message for each node specified, so the sending to each node will be handled individually.
```yaml
data : data here in byte format
```
The actual data in the message. This is the field where we put the returned data in a reply message. The data field are of type []byte.
```yaml
method : REQCliCommand
```
What request method type to use, like REQCliCommand, REQHttpGet..
```yaml
methodArgs :
- "bash"
- "-c"
- |
echo "this is a test"
echo "and some other test"
```
Additional arguments that might be needed when executing the method. Can be f.ex. an ip address if it is a tcp sender, or the actual shell command to execute in a cli.
```yaml
replyMethod : REQToFile
```
ReplyMethod, is the method to use for the reply message. By default the reply method will be set to log to file, but you can override it setting your own here.
```yaml
methodArgs :
- "bash"
- "-c"
- |
echo "this is a test"
```
Additional arguments that might be needed when executing the reply method. Can be f.ex. an ip address if it is a tcp sender, or the shell command to execute in a cli session.
replyMethodArgs :
```yaml
fromNode Node : "node2"
```
From what node the message originated. This field is automatically filled by ctrl when left blanc, so that when a message are sent from a node the user don't have to worry about getting eventual repply messages back.
Setting a
```yaml
ACKTimeout: 10
```
ACKTimeout for waiting for an Ack message in seconds.
If the ACKTimeout value is set to 0 the message will become an No Ack message. With No Ack messages we will not wait for an Ack, nor will the receiver send an Ack, and we will never try to resend a message.
```yaml
retryWait : 60
```
RetryWait specifies the time in seconds to wait between retries. This value is added to the ACKTimeout and to make the total time before retrying to sending a message. A usecase can be when you want a low ACKTimeout, but you want to add more time between the retries to avoid spamming the receiving node.
```yaml
retries : 3
```
How many times we should try to resend the message if no ACK was received.
```yaml
replyACKTimeout int `json:"replyACKTimeout" yaml:"replyACKTimeout"`
```
The ACK timeout used with reply messages. If not specified the value of `ACKTimeout` will be used.
```yaml
replyRetries int `json:"replyRetries" yaml:"replyRetries"`
```
The number of retries for trying to resend a message for reply messages.
```yaml
methodTimeout : 10
```
Timeout for how long a method should be allowed to run before it is timed out.
```yaml
replyMethodTimeout : 10
```
Timeout for how long a method should be allowed to run before it is timed out, but for the reply message.
```yaml
directory string `json:"directory" yaml:"directory"`
```
Specify the directory structure to use when saving the result data for a repply message.
- When the values are comma separated like `"syslog","metrics"` a syslog folder with a subfolder metrics will be created in the directory specified with startup env variable `SUBSCRIBER_DATA_FOLDER`.
- Absolute paths can also be used if you want to save the result somewhere else on the system, like "/etc/myservice".
```yaml
fileName : myfile.conf
```
The fileName field are used together with the directory field mentioned earlier to create a full path for where to write the resulting data of a repply message.
```yaml
schedule : [2,5]
```
Schedule are used for scheduling the method of messages to be executed several times. The schedule is defined as an array of two values, where the first value defines how often the schedule should trigger a run in seconds, and the second value is for how long the schedule are allowed to run in seconds total. `[2,5]` will trigger the method to be executed again every 2 seconds, but when a total time of 5 seconds have passed it will stop scheduling.

30
doc/src/core_messaging_overview.md Normal file
View file

@ -0,0 +1,30 @@
# Messaging
The handling of all messages is done by spawning up a process for handling the message in it's own thread. This allows us to keep the state of each **individual message level** both in regards to ACK's, error handling, send retries, and reruns of methods for a message if the first run was not successful.
All messages processed by a publisher will be written to a log file after they are processed, with all the information needed to recreate the same message if needed, or it can be used for auditing.
All handling down to the process and message level are handled concurrently. So if there are problems handling one message sent to a node on a subject it will not affect the messages being sent to other nodes, or other messages sent on other subjects to the same host.
Message types of both **ACK** and **NACK**, so we can decide if we want or don't want an Acknowledge if a message was delivered succesfully.
Example: We probably want an **ACK** when sending some **REQCLICommand** to be executed, but we don't care for an acknowledge **NACK** when we send an **REQHello** event.
If a message are **ACK** or **NACK** type are defined by the value of the **ACKTimeout** for each individual message:
- **ACKTimeout** set to 0 will make the message become a **NACK** message.
- **ACKTimeout** set to >=1 will make the message become an **ACK** message.
If ACK is expected, and not received, then the message will be retried the amount of time defined in the **retries** field of a message. If numer of retries are reached, and still no ACK received, the message will be discared, and a log message will be sent to the log server.
To make things easier, all timeouts used for messages can be set with env variables or flags at startup of ctrl. Locally specified timeout directly in a message will override the startup values, and can be used for more granularity when needed.
## Example of message flow:
<style>
img {
background-color: #FFFFFF;
}
</style>
</head>
<body>
<p align="center"><img src="https://github.com/postmannen/ctrl/blob/main/doc/message-flow.svg?raw=true" /></p>
</body>