Skip to content

Commit

Permalink
chore: add individual dump monitor commands
Browse files Browse the repository at this point in the history
  • Loading branch information
marianozunino committed Oct 10, 2024
1 parent 4e65025 commit cb069ad
Show file tree
Hide file tree
Showing 11 changed files with 439 additions and 123 deletions.
119 changes: 74 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,86 +9,115 @@
```

## Installation
To install goq, run:
To install GOQ, run:

```bash
go install github.com/marianozunino/goq@latest
```

Or download the binary from the [GitHub releases](https://github.com/marianozunino/goq/releases) (Note: This link is hypothetical and should be updated with the actual releases page)
Or download the binary from the [GitHub releases](https://github.com/marianozunino/goq/releases)

## Usage
```
Usage:
goq [flags]
goq [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
configure Create a sample configuration file
dump Dump RabbitMQ messages to a file
help Help about any command
monitor Create a temporary queue and consume messages from it having the specified routing keys
update Update GOQ to the latest version
version Print the version number of GOQ
Flags:
--config string config file (default is $XDG_CONFIG_HOME/goq/goq.yaml)
-u, --url string RabbitMQ URL (e.g., localhost:5672)
-s, --amqps Use AMQPS instead of AMQP
--config string Config file (default is $XDG_CONFIG_HOME/goq/goq.yaml)
-e, --exchange string RabbitMQ exchange name
-q, --queue string RabbitMQ queue name
-m, --file-mode string File mode (append or overwrite) (default "overwrite")
-h, --help Help for GOQ
-o, --output string Output file name (default "messages.txt")
-s, --amqps Use AMQPS instead of AMQP
-v, --virtualhost string RabbitMQ virtual host
-k, --skip-tls-verify Skip TLS certificate verification (insecure)
-a, --auto-ack Automatically acknowledge messages
-m, --file-mode string File mode (append or overwrite) (default "overwrite")
-c, --stop-after-consume Stop consuming after getting all messages from the queue
-h, --help help for goq
-u, --url string RabbitMQ URL (e.g., localhost:5672)
-v, --virtualhost string RabbitMQ virtual host
-p, --pretty-print Pretty print JSON messages
Use "goq [command] --help" for more information about a command.
```

### Examples
1. Dump messages from a local RabbitMQ server:
```
goq -u localhost:5672 -q my_queue
```
2. Use AMQPS with a specific virtual host:
```
goq -u rabbitmq.example.com:5671 -q important_queue -s -v my_vhost
```
3. Specify an exchange and output file:
### Example Commands
1. **Dump messages from a local RabbitMQ server**:
```bash
goq dump -u localhost:5672 -q my_queue
```
goq -u localhost:5672 -e my_exchange -q my_queue -o output.txt
```
4. Automatically acknowledge messages and stop after consuming:

2. **Monitor messages with specific routing keys**:
```bash
goq monitor -u localhost:5672 -r routing_key1,routing_key2,routing_key_pattern.#
```
goq -u localhost:5672 -q my_queue -a -c

3. **Use AMQPS with a specific virtual host**:
```bash
goq dump -u rabbitmq.example.com:5671 -q important_queue -s -v my_vhost
```
5. Append to an existing output file:

4. **Automatically acknowledge messages and stop after consuming**:
```bash
goq dump -u localhost:5672 -q my_queue -a -c
```
goq -u localhost:5672 -q my_queue -m append -o existing_output.txt

5. **Append to an existing output file**:
```bash
goq dump -u localhost:5672 -q my_queue -m append -o existing_output.txt
```

## Configuration
goq can be configured using a YAML file. By default, it looks for a configuration file at `$XDG_CONFIG_HOME/goq/goq.yaml`.
GOQ can be configured using a YAML file. By default, it looks for a configuration file at `$XDG_CONFIG_HOME/goq/goq.yaml`.

Example configuration file:
### Example Configuration File
```yaml
url: localhost:5672
exchange: my_exchange
queue: my_queue
output: messages.txt
amqps: false
virtualhost: /
skip-tls-verify: false
auto-ack: false
file-mode: overwrite
stop-after-consume: false
# Configuration for rabbitmq-dumper

# Use AMQPS instead of AMQP
amqps: true

# Skip TLS certificate verification (insecure)
skip-tls-verify: true

# RabbitMQ server URL
url: "127.0.0.1:5672"

# RabbitMQ exchange name, leave empty if not used
exchange: "some_exchange"

# RabbitMQ virtual host, leave empty if not used
virtualhost: "some-virtual-host"

# Output file name (relative to the current working directory)
output: "messages.txt"

# File mode (append or overwrite)
file-mode: "overwrite"

# Pretty print JSON messages
pretty-print: true
```
When using a config file the flags will override the values in the config file. So it is usefull to use the config file in combination with the flags,
When using a config file, the flags will override the values in the config file. It's useful to combine the config file with command-line flags for more flexibility.
## How Does goq Work?
## How Does GOQ Work?
1. **Connection**: Establishes a connection to the specified RabbitMQ server using either AMQP or AMQPS.
2. **Queue Binding**: Binds to the specified queue and exchange (if provided).
3. **Message Consumption**: Begins consuming messages from the queue.
4. **File Writing**: Writes consumed messages to the specified output file.
5. **Acknowledgment**: Acknowledges messages based on the auto-ack setting.
6. **Monitoring**: Displays progress of consumed messages.
6. **Monitoring**: Displays the progress of consumed messages.
7. **Termination**: Stops consuming messages based on user input (CTRL+C) or the stop-after-consume flag.
## Contributing
Contributions to goq are welcome! Please feel free to submit pull requests, create issues for bugs and feature requests, or contribute to the documentation.
Contributions to GOQ are welcome! Please feel free to submit pull requests, create issues for bugs and feature requests, or contribute to the documentation.
## License
goq is released under the MIT License. See the LICENSE file for more details.
GOQ is released under the MIT License. See the LICENSE file for more details.
30 changes: 14 additions & 16 deletions cmd/configure.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,13 @@ import (

var configureCmd = &cobra.Command{
Use: "configure",
Short: "Create a sample configuration file",
Long: `Create a sample configuration file for rabbitmq-dumper in the default location.`,
Run: runConfigure,
Short: "Generate a sample configuration file for goq.",
Long: `Generate a sample configuration file for goq in the default location.
This file provides default settings for RabbitMQ connections and message handling, which can be customized for your environment.
The configuration file simplifies using the tool by predefining connection parameters, output settings, and other preferences.`,
Example: `goq configure`,
GroupID: "available-commands",
Run: runConfigure,
}

func init() {
Expand All @@ -46,35 +50,29 @@ func init() {
func runConfigure(cmd *cobra.Command, args []string) {
exampleConfig := `# Configuration for rabbitmq-dumper
# RabbitMQ server URL
url: "localhost:5672"
# RabbitMQ exchange name
exchange: "my_exchange"
# RabbitMQ queue name
queue: "my_queue"
# Output file name
output: "messages.txt"
# Use AMQPS instead of AMQP
amqps: false
# RabbitMQ server URL
url: "localhost:5672"
# RabbitMQ virtual host
virtualhost: ""
# Skip TLS certificate verification (insecure)
skip-tls-verify: false
# Automatically acknowledge messages
auto-ack: false
# Output file name
output: "messages.txt"
# File mode (append or overwrite)
file-mode: "overwrite"
# Stop consuming after getting all messages from the queue
stop-after-consume: false
# Pretty print JSON messages
pretty-print: false
`

configPath, err := xdg.ConfigFile("goq/goq.yaml")
Expand Down
79 changes: 79 additions & 0 deletions cmd/dump.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
/*
Copyright © 2024 Mariano Zunino <[email protected]>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/

package cmd

import (
"fmt"

app "github.com/marianozunino/goq/internal"
"github.com/marianozunino/goq/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)

func NewDumpCmd() *cobra.Command {
var queue string
var autoAck bool
var stopAfterConsume bool

dumpCmd := &cobra.Command{
Use: "dump",
Short: "Dump messages from a RabbitMQ queue to a file.",
Long: `Dump messages from the specified RabbitMQ queue to a file.
This command provides options for automatically acknowledging messages, controlling when consumption stops, and configuring file output behavior.
Messages can be captured from an AMQP or AMQPS RabbitMQ server, with flexible TLS and virtual host settings.`,
Example: `goq dump -q my_queue -o output.txt -a -c`,
GroupID: "available-commands",
RunE: func(cmd *cobra.Command, args []string) error {
cfg := config.New(

config.WithRabbitMQURL(fmt.Sprintf("%s://%s/%s", getProtocol(), viper.GetString("url"), viper.GetString("virtualhost"))),
config.WithExchange(viper.GetString("exchange")),
config.WithQueue(queue),
config.WithOutputFile(viper.GetString("output")),
config.WithUseAMQPS(viper.GetBool("amqps")),
config.WithVirtualHost(viper.GetString("virtualhost")),
config.WithSkipTLSVerification(viper.GetBool("skip-tls-verify")),
config.WithAutoAck(viper.GetBool("auto-ack")),
config.WithFileMode(viper.GetString("file-mode")),
config.WithStopAfterConsume(viper.GetBool("stop-after-consume")),
config.WithPrettyPrint(viper.GetBool("pretty-print")),
)

return app.Dump(cfg)
},
}

dumpCmd.Flags().SortFlags = false
dumpCmd.Flags().StringVarP(&queue, "queue", "q", "", "RabbitMQ queue name")
dumpCmd.Flags().BoolVarP(&autoAck, "auto-ack", "a", false, "Auto ack messages")
dumpCmd.Flags().BoolVarP(&stopAfterConsume, "stop-after-consume", "c", false, "Stop after consuming messages")

dumpCmd.MarkFlagRequired("queue")

return dumpCmd
}

func init() {
rootCmd.AddCommand(NewDumpCmd())
}
72 changes: 72 additions & 0 deletions cmd/monitor.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
Copyright © 2024 Mariano Zunino <[email protected]>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
package cmd

import (
"fmt"

app "github.com/marianozunino/goq/internal"
"github.com/marianozunino/goq/internal/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)

var routingKeys []string

func NewMonitorCmd() *cobra.Command {
monitorCmd := &cobra.Command{
Use: "monitor",
Aliases: []string{"mon"},
GroupID: "available-commands",
Short: "Monitor RabbitMQ messages using routing keys and a temporary queue.",
Long: `Monitor RabbitMQ messages by consuming from a temporary queue that listens to specified routing keys.
This command captures and dumps the received messages to a file for analysis or processing.`,
Example: `goq monitor -r "key1,key2" -o output.txt -s -v my_vhost`,
RunE: func(cmd *cobra.Command, args []string) error {
cfg := config.New(
config.WithRabbitMQURL(fmt.Sprintf("%s://%s/%s", getProtocol(), viper.GetString("url"), viper.GetString("virtualhost"))),
config.WithExchange(viper.GetString("exchange")),
config.WithOutputFile(viper.GetString("output")),
config.WithUseAMQPS(viper.GetBool("amqps")),
config.WithVirtualHost(viper.GetString("virtualhost")),
config.WithSkipTLSVerification(viper.GetBool("skip-tls-verify")),
config.WithAutoAck(viper.GetBool("auto-ack")),
config.WithFileMode(viper.GetString("file-mode")),
config.WithPrettyPrint(viper.GetBool("pretty-print")),
config.WithRoutingKeys(routingKeys),
)

return app.Monitor(cfg)
},
}

// Allow to pass an array of routing keys to monitor
monitorCmd.Flags().SortFlags = false
monitorCmd.Flags().StringSliceVarP(&routingKeys, "routing-keys", "r", nil, "List of routing keys to monitor")
monitorCmd.MarkFlagRequired("routing-keys")

return monitorCmd
}

func init() {
rootCmd.AddCommand(NewMonitorCmd())
}
Loading

0 comments on commit cb069ad

Please sign in to comment.