mirrors/ctrl
1
0
Fork 0
mirror of https://github.com/postmannen/ctrl.git synced 2025-01-18 21:59:30 +00:00
Code Activity

updated todo

Browse source
This commit is contained in:
postmannen 2021-09-09 09:54:33 +02:00
parent 3dba156820
commit 2085034e47
2 changed files with 129 additions and 68 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

185
README.md
View file

@ -139,14 +139,10 @@ A default config file will be created at first startup if one does not exist, wi
If new values are provided via flags they will take precedence over the ones currently in the config file, and they will also replace the current value in the config file, making it the default for the next restart. If new values are provided via flags they will take precedence over the ones currently in the config file, and they will also replace the current value in the config file, making it the default for the next restart.
The only exception from the above are the `startSubX` flags which got one extra value that can be used which is the value `RST` for Reset. This will disable the specified subscriber, and also null out the array for which Nodes the subscriber will allow traffic from.
The config file can also be edited directly, making the use of flags not needed. The config file can also be edited directly, making the use of flags not needed.
If just getting back to standard default for all config options needed, then delete the current config file, restart Steward, and a new config file with all the options set to it's default values will be created. If just getting back to standard default for all config options needed, then delete the current config file, restart Steward, and a new config file with all the options set to it's default values will be created.
TIP: Most likely the best way to control how the service should behave and what is started is to start Steward the first time so it creates the default config file. Then stop the service, edit the config file and change the defaults needed. Then start the service again.
### Request Methods ### Request Methods
#### REQCliCommand #### REQCliCommand
@ -213,7 +209,15 @@ Write the output of the reply message to a log file specified with the `director
### Build and Run ### Build and Run
clone the repository, then cd `./steward/cmd` and do `go build -o steward`, and run the application with `./steward --help` Steward is written in go, so you need Go installed to compile it. You can get go at <https://golang.org/dl/>.
clone the repository `git clone https://github.com/RaaLabs/steward.git`.
Then cd `./steward/cmd` and do `go build -o steward`._1FJeI4_rrR6mY_Y496rsi5
Run the application with `env CONFIG_FOLDER </myconfig/folder/here> ./steward`
You can get all the options with `./steward --help`
### Options for running ### Options for running
@ -222,46 +226,69 @@ The location of the config file are given via an env variable at startup (defaul
`env CONFIG_FOLDER </myconfig/folder/here>` `env CONFIG_FOLDER </myconfig/folder/here>`
```text ```text
-brokerAddress string // The configuration folder on disk
the address of the message broker (default "127.0.0.1:4222") ConfigFolder string
-centralNodeName string // The folder where the socket file should live
The name of the central node to receive messages published by this node (default "central") SocketFolder string
-defaultMessageRetries int // TCP Listener for sending messages to the system
default amount of retries that will be done before a message is thrown away, and out of the system (default 3) TCPListener string
-defaultMessageTimeout int // The folder where the database should live
default message timeout in seconds. This can be overridden on the message level (default 5) DatabaseFolder string
-nodeName string // some unique string to identify this Edge unit
some unique string to identify this Edge unit (default "central") NodeName string
-profilingPort string // the address of the message broker
The number of the profiling port BrokerAddress string
-promHostAndPort string // nats connect retry
host and port for prometheus listener, e.g. localhost:2112 (default ":2112") NatsConnectRetryInterval int
-startPubREQHello int // The number of the profiling port
Make the current node send hello messages to central at given interval in seconds ProfilingPort string
-startSubREQCliCommand value // host and port for prometheus listener, e.g. localhost:2112
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. PromHostAndPort string
-startSubREQErrorLog value // set to true if this is the node that should receive the error log's from other nodes
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. DefaultMessageTimeout int
-startSubREQHello value // default amount of retries that will be done before a message is thrown away, and out of the system
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. DefaultMessageRetries int
-startSubREQHttpGet value // Publisher data folder
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. SubscribersDataFolder string
-startSubREQPing value // central node to receive messages published from nodes
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. CentralNodeName string
-startSubREQPong value // Path to the certificate of the root CA
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. RootCAPath string
-startSubREQTailFile value // Full path to the NKEY's seed file
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. NkeySeedFile string
-startSubREQToConsole value // The host and port to expose the data folder
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. ExposeDataFolder string
-startSubREQToFile value // Timeout for error messages
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. ErrorMessageTimeout int
-startSubREQToFileAppend value // Retries for error messages.
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. ErrorMessageRetries int
-startSubREQnCliCommand value // Make the current node send hello messages to central at given interval in seconds
Specify comma separated list for nodes to allow messages from. Use "*" for from all. Value RST will turn off subscriber. StartPubREQHello int
-subscribersDataFolder string // Start the central error logger.
The data folder where subscribers are allowed to write their data if needed (default "./var") // Takes a comma separated string of nodes to receive from or "*" for all nodes.
StartSubREQErrorLog bool
// Subscriber for hello messages
StartSubREQHello bool
// Subscriber for text logging
StartSubREQToFileAppend bool
// Subscriber for writing to file
StartSubREQToFile bool
// Subscriber for Echo Request
StartSubREQPing bool
// Subscriber for Echo Reply
StartSubREQPong bool
// Subscriber for CLICommandRequest
StartSubREQCliCommand bool
// Subscriber for REQnCliCommand
StartSubREQnCliCommand bool
// Subscriber for REQToConsole
StartSubREQToConsole bool
// Subscriber for REQHttpGet
StartSubREQHttpGet bool
// Subscriber for tailing log files
StartSubREQTailFile bool
// Subscriber for continously delivery of output from cli commands.
StartSubREQnCliCommandCont bool
``` ```
### How to Run ### How to Run
@ -270,6 +297,8 @@ The location of the config file are given via an env variable at startup (defaul
Nkey's can be used for authentication, and you use the `nkeySeedFile` flag to specify the seed file to use. Nkey's can be used for authentication, and you use the `nkeySeedFile` flag to specify the seed file to use.
Read more in the sections below on how to generate nkey's.
#### nats-server (the message broker) #### nats-server (the message broker)
The broker for messaging is Nats-server from <https://nats.io>. Download, run it, and use the `-brokerAddress` flag on Steward to point to it. The broker for messaging is Nats-server from <https://nats.io>. Download, run it, and use the `-brokerAddress` flag on Steward to point to it.
@ -344,15 +373,15 @@ More example configurations for the nats-server are located in the `doc` folder
To set the location of the config folder other than default, you should use the ENV variable `CONFIG_FOLDER`. To set the location of the config folder other than default, you should use the ENV variable `CONFIG_FOLDER`.
`env CONFIG_FOLDER=./etc/ ./steward --node="central"` `env CONFIG_FOLDER=./etc/ ./steward --nodeName="central" --centralNodeName="central"`
Using default config folder location on some central server which will act as your command and control server. Or using default config folder location on some central server which will act as your command and control server.
`./steward --node="central"` `./steward --nodeName="central" --centralNodeName="central"`
One the nodes out there One the nodes out there
`./steward --node="ship1"` & `./steward --node="ship1"` and so on. `./steward ./steward --nodeName="ship1" --centralNodeName="central"` & `./steward --node="ship2"` and so on.
Use the `-help` flag to get all possibilities. Use the `-help` flag to get all possibilities.
@ -364,18 +393,20 @@ And start another node that will be managed via central.
`./steward --nodeName="ship1" -startPubREQHello=200 --centralNodeName="central" -promHostAndPort=":12112" -brokerAddress="127.0.0.1:4222"` `./steward --nodeName="ship1" -startPubREQHello=200 --centralNodeName="central" -promHostAndPort=":12112" -brokerAddress="127.0.0.1:4222"`
#### Start subscriber flags
The start subscribers flags take a string value of which nodes that it will process messages from. Since using a flag to set a value automatically sets that value also in the config file, a value of RST can be given to turn off the subscriber.
### Message fields explanation ### Message fields explanation
```go ```go
// The node to send the message to // The node to send the message to
toNode toNode
// The actual data in the message // ToNodes to specify several hosts to send message to in the
// form of an slice/array.
toNodes
// The actual data in the message. This is typically where we
// specify the cli commands to execute on a node, and this is
// also the field where we put the returned data in a reply
// message.
data data
// Method, what is this message doing, etc. CLI, syslog, etc. // Method, what request type to use, like REQCliCommand, REQHttpGet..
method method
// ReplyMethod, is the method to use for the reply message. // ReplyMethod, is the method to use for the reply message.
// By default the reply method will be set to log to file, but // By default the reply method will be set to log to file, but
@ -406,12 +437,16 @@ operation
### How to send a Message ### How to send a Message
The API for sending a message from one node to another node is by pasting a structured JSON object into the socket file file called `steward.sock` which by default lives in the `./tmp` directory. This file will be read continously, and when updated the content will be picked up, umarshaled, and if OK it will be sent a message to the node specified in the `toNode` field. The API for sending a message from one node to another node is by pasting a structured JSON object into the socket file file called `steward.sock` which by default lives in the `./tmp` directory, or by starting the TCPListener, or via http (not implented yet). The message will be picked up, umarshaled, and if OK it will be sent as a message to the node specified in the `toNode` field.
The `method` is what defines what the event will do. The `method` is what defines what the event will do.
The `Operation` field is a little bit special. This field is used with the `REQOpCommand` to specify what operation command to run, and also it's arguments. The `Operation` field is a little bit special. This field is used with the `REQOpCommand` to specify what operation command to run, and also it's arguments.
#### Send to socket with netcat
`nc -U ./tmp/steward.sock < myMessage.json`
#### The current `operation`'s that are available are #### The current `operation`'s that are available are
To stop a process of a specific type on a node. To stop a process of a specific type on a node.
@ -486,12 +521,13 @@ NB: Both the keys and the values used are case sensitive.
#### Sending a command from one Node to Another Node #### Sending a command from one Node to Another Node
Example JSON for appending a message of type command into the `socket` file ##### Example JSON for appending a message of type command into the `socket` file
```json ```json
[ [
{ {
"directory":"/var/steward/cli-command/executed-result", "directory":"/var/steward/cli-command/executed-result",
"fileName": "some.log",
"toNode": "ship1", "toNode": "ship1",
"data": ["bash","-c","sleep 3 & tree ./"], "data": ["bash","-c","sleep 3 & tree ./"],
"method":"REQCliCommand", "method":"REQCliCommand",
@ -502,12 +538,13 @@ Example JSON for appending a message of type command into the `socket` file
] ]
``` ```
To specify more messages at once do ##### Specify more messages at once do
```json ```json
[ [
{ {
"directory":"cli-command-executed-result", "directory":"cli-command-executed-result",
"fileName": "some.log",
"toNode": "ship1", "toNode": "ship1",
"data": ["bash","-c","sleep 3 & tree ./"], "data": ["bash","-c","sleep 3 & tree ./"],
"method":"REQCliCommand", "method":"REQCliCommand",
@ -517,6 +554,7 @@ To specify more messages at once do
}, },
{ {
"directory":"cli-command-executed-result", "directory":"cli-command-executed-result",
"fileName": "some.log",
"toNode": "ship2", "toNode": "ship2",
"data": ["bash","-c","sleep 3 & tree ./"], "data": ["bash","-c","sleep 3 & tree ./"],
"method":"REQCliCommand", "method":"REQCliCommand",
@ -527,7 +565,26 @@ To specify more messages at once do
] ]
``` ```
To send a Op Command message for process listing with custom timeout and amount of retries ##### Send the same message to several hosts by using the toHosts field
```json
[
{
"directory": "httpget",
"fileName": "finn.no.html",
"toNode": "",
"toNodes": ["central","ship2"],
"data": ["https://finn.no"],
"method":"REQHttpGet",
"replyMethod":"REQToFile",
"ACKTimeout":5,
"retries":3,
"methodTimeout": 5
}
]
```
##### Send an Op Command message for process listing with custom timeout and amount of retries
```json ```json
[ [
@ -544,12 +601,12 @@ To send a Op Command message for process listing with custom timeout and amount
"retries":3, "retries":3,
"replyACKTimeout":3, "replyACKTimeout":3,
"replyRetries":3, "replyRetries":3,
"MethodTimeout": 7 "methodTimeout": 7
} }
] ]
``` ```
To send and Op Command to stop a subscriber on a node ##### Send and Op Command to stop a subscriber on a node
```json ```json
[ [
@ -571,12 +628,12 @@ To send and Op Command to stop a subscriber on a node
"retries":3, "retries":3,
"replyACKTimeout":3, "replyACKTimeout":3,
"replyRetries":3, "replyRetries":3,
"MethodTimeout": 7 "methodTimeout": 7
} }
] ]
``` ```
To send and Op Command to start a subscriber on a node ##### Send and Op Command to start a subscriber on a node
```json ```json
[ [
@ -597,7 +654,7 @@ To send and Op Command to start a subscriber on a node
"retries":3, "retries":3,
"replyACKTimeout":3, "replyACKTimeout":3,
"replyRetries":3, "replyRetries":3,
"MethodTimeout": 7 "methodTimeout": 7
} }
] ]
``` ```

12
message_and_subject.go
View file

@ -14,13 +14,17 @@ import (
type Message struct { type Message struct {
// The node to send the message to. // The node to send the message to.
ToNode Node `json:"toNode" yaml:"toNode"` ToNode Node `json:"toNode" yaml:"toNode"`
// ToNodes to specify several hosts to send message to. // ToNodes to specify several hosts to send message to in the
// form of an slice/array.
ToNodes []Node `json:"toNodes,omitempty" yaml:"toNodes,omitempty"` ToNodes []Node `json:"toNodes,omitempty" yaml:"toNodes,omitempty"`
// The Unique ID of the message // The Unique ID of the message
ID int `json:"id" yaml:"id"` ID int `json:"id" yaml:"id"`
// The actual data in the message // The actual data in the message. This is typically where we
// specify the cli commands to execute on a node, and this is
// also the field where we put the returned data in a reply
// message.
Data []string `json:"data" yaml:"data"` Data []string `json:"data" yaml:"data"`
// Method, what is this message doing, etc. CLI, syslog, etc. // Method, what request type to use, like REQCliCommand, REQHttpGet..
Method Method `json:"method" yaml:"method"` Method Method `json:"method" yaml:"method"`
// ReplyMethod, is the method to use for the reply message. // ReplyMethod, is the method to use for the reply message.
// By default the reply method will be set to log to file, but // By default the reply method will be set to log to file, but
@ -43,7 +47,7 @@ type Message struct {
// For example "syslog","metrics", or "metrics/mysensor" // For example "syslog","metrics", or "metrics/mysensor"
// The type is typically used in the handler of a method. // The type is typically used in the handler of a method.
Directory string `json:"directory" yaml:"directory"` Directory string `json:"directory" yaml:"directory"`
// FileName is used to be able to set a wanted extension // FileName is used to be able to set a wanted name
// on a file being saved as the result of data being handled // on a file being saved as the result of data being handled
// by a method handler. // by a method handler.
FileName string `json:"fileName" yaml:"fileName"` FileName string `json:"fileName" yaml:"fileName"`