Skip to content

Commit

Permalink
refactor: rename AtomSpaceNode -> DistributedAlgorithmNode (#54)
Browse files Browse the repository at this point in the history
  • Loading branch information
Pedrobc89 authored Nov 8, 2024
1 parent 896268b commit 9c3c742
Show file tree
Hide file tree
Showing 26 changed files with 186 additions and 159 deletions.
4 changes: 2 additions & 2 deletions BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
cc_library(
name = "atomspacenode",
name = "distributedalgorithmnode",
srcs = [],
deps = ["//src/atom_space_node:atom_space_node_lib"],
deps = ["//src/distributed_algorithm_node:distributed_algorithm_node_lib"],
visibility = ["//visibility:public"],
)

4 changes: 2 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,12 @@ find_package(nanobind CONFIG REQUIRED)
include_directories(
${CMAKE_CURRENT_SOURCE_DIR}/bazel_assets
${CMAKE_CURRENT_SOURCE_DIR}/src/commons
${CMAKE_CURRENT_SOURCE_DIR}/src/atom_space_node)
${CMAKE_CURRENT_SOURCE_DIR}/src/distributed_algorithm_node)

file(GLOB_RECURSE headers
${CMAKE_CURRENT_SOURCE_DIR}/bazel_assets/*.h
${CMAKE_CURRENT_SOURCE_DIR}/src/commons/*.h
${CMAKE_CURRENT_SOURCE_DIR}/src/atom_space_node/*.h)
${CMAKE_CURRENT_SOURCE_DIR}/src/distributed_algorithm_node/*.h)

nanobind_add_module(
${EXTENSION_NAME}
Expand Down
20 changes: 10 additions & 10 deletions Design.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# AtomSpaceNode Design
# DistributedAlgorithmNode Design

This is a design proposal of the Distributed AtomSpace (DAS) Node.
This is a design proposal of the Distributed Atom Space (DAS) Algorithm Node.

Initial design will focus on Messaging and Leader Election.

## Goals

- Ensure the AtomSpaceNode is resilient to network failures.
- Ensure the AtomSpaceNode is resilient to leader failures.
- Ensure the AtomSpaceNode is resilient to job failures.
- Ensure the AtomSpaceNode will satisfy the use case.
- Ensure the `DistributedAlgorithmNode` is resilient to network failures.
- Ensure the `DistributedAlgorithmNode` is resilient to leader failures.
- Ensure the `DistributedAlgorithmNode` is resilient to job failures.
- Ensure the `DistributedAlgorithmNode` will satisfy the use case.
- Ensure Nodes run inside docker containers.
- Ensure speedup of at least 70% \* N considering the
execution in a network with N (1 < N < 6) equally resourced NODEs against the
Expand All @@ -25,9 +25,9 @@ job is requested.
- USER: a person using the software we provide to execute JOBs in a network of
NODEs.
- NODE: a server process running on a Docker container encompassing all
components required to run a AtomSpaceNode.
components required to run a DistributedAlgorithmNode.
- JOB: a data structure designed to contain a script in one of the script
languages supported by AtomSpaceNode. This script contains all the code required to
languages supported by DistributedAlgorithmNode. This script contains all the code required to
execute a given task.
- MESSAGE: a data structure to encapsulate pieces of information we want to
transport from one NODE to another.
Expand Down Expand Up @@ -62,7 +62,7 @@ job is requested.
and ordered delivery.
- The messaging system will abstract the underlying network topology.
- The messaging layer will be an external library. That way we can easily
change it, without changing the code of the AtomSpaceNode.
change it, without changing the code of the DistributedAlgorithmNode.

#### Considered Topologies

Expand Down Expand Up @@ -210,7 +210,7 @@ Perform a single query to the remote DAS Server, process all the results and
perform some extra computation on each result in order to evaluate each
result's quality.

JOB should be defined as a script in some programming language. AtomSpaceNode should
JOB should be defined as a script in some programming language. DistributedAlgorithmNode should
be able to support multiple programming languages here so the design must be
flexible. Initially we'll support only Python scripts doing queries to a remote
DAS Server.
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Hyperon Distributed Atomspace Node
# Hyperon Distributed Atom Space Algorithm Node

The Distributed AtomSpace Node (aka DAS Node) is a component of DAS
The Distributed Atom Space Algorithm Node (aka DAS Node) is a component of DAS
(<https://github.com/singnet/das>) which allows the implementation of
distributed algorithms using one or more DAS as shared knowledge base.

Expand All @@ -13,9 +13,9 @@ exchanging built in the component.

## Features

### AtomSpaceNode
### DistributedAlgorithmNode

AtomSpaceNode is an abstract class that represents a node in a network,
DistributedAlgorithmNode is an abstract class that represents a node in a network,
designed for distributed processing. Users must extend this class to implement
custom distributed networks.

Expand All @@ -25,25 +25,25 @@ transmitted. Instead, a command identifier is sent, and the receiving node
reconstructs and executes the corresponding Message object. The command format
resembles a command-line interface, with commands and arguments.

Key Points for Extending AtomSpaceNode:
Key Points for Extending DistributedAlgorithmNode:

- AtomSpaceNode builds Message objects because it inherits from
- DistributedAlgorithmNode builds Message objects because it inherits from
MessageFactory. The base class can create basic Message objects for common
tasks, such as handling new nodes joining the network. However, subclasses
should override message_factory() to handle application-specific messages.

- Message execution is threaded. If commands update shared state in
AtomSpaceNode or its subclasses, you must protect this state using mutual
DistributedAlgorithmNode or its subclasses, you must protect this state using mutual
exclusion mechanisms.

- The constructor for AtomSpaceNode requires a MessageBroker and a
- The constructor for DistributedAlgorithmNode requires a MessageBroker and a
LeadershipBroker, both of which are abstract. You must choose concrete
implementations or create your own, depending on the communication and
leadership election strategies you want to use. Custom leadership algorithms
may be needed, depending on the network topology and application
requirements.

- AtomSpaceNode has several pure virtual methods that must be implemented by
- DistributedAlgorithmNode has several pure virtual methods that must be implemented by
subclasses. These methods handle fundamental tasks, such as leadership
elections and notifying nodes when new peers join the network.

Expand All @@ -61,7 +61,7 @@ invoke its act() method.

LeadershipBroker defines the API for leader election in the network. Users
typically don't interact with this class directly; it's managed by the
AtomSpaceNode.
DistributedAlgorithmNode.

#### SingleMasterServer

Expand All @@ -72,7 +72,7 @@ clients.
### MessageBroker

MessageBroker defines the API for the communication layer between nodes. Users
of AtomSpaceNode don't interact with MessageBroker directly.
of DistributedAlgorithmNode don't interact with MessageBroker directly.

Currently there are two implementations: RAM and GRPC

Expand Down
4 changes: 2 additions & 2 deletions examples/simple_node.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from hyperon_das_node import AtomSpaceNode, Message, LeadershipBrokerType, MessageBrokerType
from hyperon_das_node import DistributedAlgorithmNode, Message, LeadershipBrokerType, MessageBrokerType

class PrintMessage(Message):
def __init__(self, content: str):
Expand All @@ -9,7 +9,7 @@ def act(self, node: "SimpleNode") -> None:
# ideally we should call a node.method in here
node.print_content(self.content)

class SimpleNode(AtomSpaceNode):
class SimpleNode(DistributedAlgorithmNode):
def __init__(self, node_id: str, is_server: bool) -> None:
super().__init__(
node_id,
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
name = "hyperon_das_node"
version = "0.0.1"
description = "Distribuited Atom Space Node for Hyperon"
description = "Distrbuted Algorithm Node for Hyperon"
readme = "README.md"
requires-python = ">=3.10"
authors = [
Expand Down
6 changes: 6 additions & 0 deletions scripts/cpp_build_cmd.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ cp -r bazel-src/external/com_github_grpc_grpc/include/grpc $output/
cp -r bazel-src/external/com_github_grpc_grpc/include/grpcpp/ $output/
cp -r bazel-src/external/com_google_absl/absl/ $output/
cp -r bazel-src/external/com_google_protobuf/src/google/ $output/

# TODO: Once das-proto is updated, update atom_space_node to distributed_algorithm_node

# cp bazel-bin/external/com_github_singnet_das_proto/distributed_algorithm_node.grpc.pb.h $output/
# cp bazel-bin/external/com_github_singnet_das_proto/distributed_algorithm_node.pb.h $output/
cp bazel-bin/external/com_github_singnet_das_proto/atom_space_node.grpc.pb.h $output/
cp bazel-bin/external/com_github_singnet_das_proto/atom_space_node.pb.h $output/

cp bazel-bin/external/com_github_singnet_das_proto/common.pb.h $output/
3 changes: 2 additions & 1 deletion src/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@ cc_binary(
linkstatic = 1,
features = ["fully_static_link"],
deps = [
"//atom_space_node:atom_space_node_lib",
"//distributed_algorithm_node:distributed_algorithm_node_lib",
"//main:main_lib",
"//commons:commons_lib",
"@com_github_singnet_das_proto//:atom_space_node_cc_grpc",
# "@com_github_singnet_das_proto//:distributed_algorithm_node_cc_grpc",
"@com_github_grpc_grpc//:grpc++",
"@com_github_grpc_grpc//:grpc++_reflection",
],
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package(default_visibility = ["//visibility:public"])

cc_library(
name = "atom_space_node_lib",
name = "distributed_algorithm_node_lib",
srcs = glob(["*.cc"]),
hdrs = glob(["*.h"]),
includes = ["."],
Expand All @@ -10,6 +10,7 @@ cc_library(
features = ["fully_static_link"],
deps = [
"@com_github_singnet_das_proto//:atom_space_node_cc_grpc",
# "@com_github_singnet_das_proto//:distributed_algorithm_node_cc_grpc",
"@com_github_grpc_grpc//:grpc++",
"@com_github_grpc_grpc//:grpc++_reflection",
"//commons:commons_lib"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#include "AtomSpaceNode.h"
#include "DistributedAlgorithmNode.h"
#include "Utils.h"

using namespace atom_space_node;
using namespace distributed_algorithm_node;
using namespace commons;

AtomSpaceNode::AtomSpaceNode(
DistributedAlgorithmNode::DistributedAlgorithmNode(
const string &node_id,
LeadershipBrokerType leadership_algorithm,
MessageBrokerType messaging_backend) {
Expand All @@ -18,14 +18,14 @@ AtomSpaceNode::AtomSpaceNode(
node_id);
}

AtomSpaceNode::~AtomSpaceNode() {
DistributedAlgorithmNode::~DistributedAlgorithmNode() {
this->graceful_shutdown();
}

// -------------------------------------------------------------------------------------------------
// Public API

void AtomSpaceNode::join_network() {
void DistributedAlgorithmNode::join_network() {
this->leadership_broker->set_message_broker(this->message_broker);
this->message_broker->join_network();
//Utils::sleep(1000);
Expand All @@ -39,46 +39,46 @@ void AtomSpaceNode::join_network() {
this->message_broker->broadcast(this->known_commands.NODE_JOINED_NETWORK, args);
}

bool AtomSpaceNode::is_leader() {
bool DistributedAlgorithmNode::is_leader() {
return (this->leader_id() == this->node_id());
}

string AtomSpaceNode::leader_id() {
string DistributedAlgorithmNode::leader_id() {
return this->leadership_broker->leader_id();
}

bool AtomSpaceNode::has_leader() {
bool DistributedAlgorithmNode::has_leader() {
return this->leadership_broker->has_leader();
}

void AtomSpaceNode::add_peer(const string &peer_id) {
void DistributedAlgorithmNode::add_peer(const string &peer_id) {
this->message_broker->add_peer(peer_id);
}

string AtomSpaceNode::node_id() {
string DistributedAlgorithmNode::node_id() {
return this->my_node_id;
}

void AtomSpaceNode::broadcast(const string &command, const vector<string> &args) {
void DistributedAlgorithmNode::broadcast(const string &command, const vector<string> &args) {
this->message_broker->broadcast(command, args);
}

void AtomSpaceNode::send(
void DistributedAlgorithmNode::send(
const string &command,
const vector<string> &args,
const string &recipient) {

this->message_broker->send(command, args, recipient);
}

std::shared_ptr<Message> AtomSpaceNode::message_factory(string &command, vector<string> &args) {
std::shared_ptr<Message> DistributedAlgorithmNode::message_factory(string &command, vector<string> &args) {
if (command == this->known_commands.NODE_JOINED_NETWORK) {
return std::shared_ptr<Message>(new NodeJoinedNetwork(args[0]));
} else {
return std::shared_ptr<Message>{};
}
}

void AtomSpaceNode::graceful_shutdown() {
void DistributedAlgorithmNode::graceful_shutdown() {
this->message_broker->graceful_shutdown();
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#ifndef _ATOM_SPACE_NODE_ATOMSPACENODE_H
#define _ATOM_SPACE_NODE_ATOMSPACENODE_H
#ifndef _DISTRIBUTED_ALGORITHM_NODE_DISTRIBUTEDALGORITHMNODE_H
#define _DISTRIBUTED_ALGORITHM_NODE_DISTRIBUTEDALGORITHMNODE_H

#include "LeadershipBroker.h"
#include "MessageBroker.h"
Expand All @@ -9,7 +9,7 @@

using namespace std;

namespace atom_space_node {
namespace distributed_algorithm_node {

/**
* Implements a node in a network used for some distributed processing.
Expand All @@ -24,17 +24,17 @@ namespace atom_space_node {
* and its code is executed. The "reference" about what type of Message is being
* exchanged is encoded like command lines with commands and its arguments.
*
* This is what you should care about when extending AtomSpaceNode:
* This is what you should care about when extending DistributedAlgorithmNode:
*
* - AtomSpaceNode is the class that build Message objects. That's why it extends
* - DistributedAlgorithmNode is the class that build Message objects. That's why it extends
* MessageFactory. This abstract class already knows how to build some basic Message
* objects for some commands which are common to all the networks (e.g. joining
* of new nodes to the network) but subclasses of AtomSpaceNode should re-implement
* of new nodes to the network) but subclasses of DistributedAlgorithmNode should re-implement
* message_factory() to build Message objects for its own distributed application.
* - Message commands are executed in separated threads. So if such commands updates some
* state variable inside AtomSpaceNode or its subclasses, this state variable should
* state variable inside DistributedAlgorithmNode or its subclasses, this state variable should
* be protected by mutual exclusion primitives.
* - AtomSpaceNode's constructor expects allocated objects for MessageBroker and
* - DistributedAlgorithmNode's constructor expects allocated objects for MessageBroker and
* LeadershipBroker. These are abstract classes which may have many concrete
* implementations. When designing your distributed network you need to decide
* which messaging system is supposed to be used to actually make the communication
Expand All @@ -43,19 +43,19 @@ namespace atom_space_node {
* wouldn't need to implement its own messaging system but it may be the case for the
* leadership election algorithm as this is highly dependent on the network topology
* and expected behaviour.
* - AtomSpaceNode has some pure virtual methods that need to be implemented. These are
* - DistributedAlgorithmNode has some pure virtual methods that need to be implemented. These are
* the methods called by basic Message objects to acomplish basic tasks like leadership
* election and notification of new nodes joning the network. The proper way to respond
* to these requests are delegated to concrete classes extending AtomSpaceNode.
* to these requests are delegated to concrete classes extending DistributedAlgorithmNode.
*/
class AtomSpaceNode : public MessageFactory {
class DistributedAlgorithmNode : public MessageFactory {

public:

/**
* Destructor.
*/
virtual ~AtomSpaceNode();
virtual ~DistributedAlgorithmNode();

/**
* Basic constructor.
Expand All @@ -65,7 +65,7 @@ class AtomSpaceNode : public MessageFactory {
* @param leadership_algorithm The concrete class to be used as leadership broker.
* @param messaging_backend The concrete class to be used as message broker.
*/
AtomSpaceNode(
DistributedAlgorithmNode(
const string &node_id,
LeadershipBrokerType leadership_algorithm,
MessageBrokerType messaging_backend
Expand Down Expand Up @@ -185,6 +185,6 @@ class AtomSpaceNode : public MessageFactory {
shared_ptr<MessageBroker> message_broker;
};

} // namespace atom_space_node
} // namespace distributed_algorithm_node

#endif // _ATOM_SPACE_NODE_ATOMSPACENODE_H
#endif // _DISTRIBUTED_ALGORITHM_NODE_DISTRIBUTEDALGORITHMNODE_H
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#include "LeadershipBroker.h"
#include "Utils.h"

using namespace atom_space_node;
using namespace distributed_algorithm_node;

// -------------------------------------------------------------------------------------------------
// Constructors and destructors
Expand Down
Loading

0 comments on commit 9c3c742

Please sign in to comment.