Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

L3-Rabbit merge #7

Merged
merged 59 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
7cb3c90
Started L3
Sergeydigl3 Apr 4, 2024
bc2809b
cmd for server / client / worker
potat-dev Apr 4, 2024
f7b17ed
commit for Max <33
potat-dev Apr 11, 2024
660ef6f
Add new data types and json binds
potat-dev Apr 11, 2024
ccc5185
start RabbitWorker
L0rdLizard Apr 11, 2024
012d437
Fix Connection* issue
potat-dev Apr 11, 2024
dffc3c2
Use int ids for func in Task
potat-dev Apr 11, 2024
990fa18
handler
L0rdLizard Apr 11, 2024
c7a12ce
Demo Client prompts ready
potat-dev Apr 12, 2024
798b30a
handler complete
L0rdLizard Apr 12, 2024
1ef3d4c
simpleMathHandler complete
L0rdLizard Apr 12, 2024
eae6c99
move functions
L0rdLizard Apr 12, 2024
53956a8
init connection
L0rdLizard Apr 12, 2024
b6e21bb
Client Ready
potat-dev Apr 12, 2024
40c4360
Merge remote-tracking branch 'origin/base/L3-Rabbit' into base/L3-Rabbit
potat-dev Apr 12, 2024
3e3bf90
Add Id to TaskRequest
potat-dev Apr 12, 2024
f165ac2
Create Clients / Workers DB
potat-dev Apr 12, 2024
e73df4d
Fix Worker type
potat-dev Apr 12, 2024
cf07363
Add very useful TODOs
potat-dev Apr 12, 2024
7a4340a
func handler ready
L0rdLizard Apr 24, 2024
d1145bd
Updated data models
potat-dev Apr 24, 2024
6d60f7d
Main server connection handler implemented
potat-dev Apr 24, 2024
b695179
Add TaskStatus enum
potat-dev Apr 24, 2024
b9e5c28
Minor fixes and formatting
potat-dev Apr 24, 2024
8d4e54f
Draft for Client and Worker handlers
potat-dev Apr 24, 2024
e45069b
Task and User databases logic ready
potat-dev Apr 25, 2024
870b197
Main Server logic ready
potat-dev Apr 25, 2024
4d67cf5
fix build server errors - make json methods inline
potat-dev Apr 25, 2024
529c1a9
change id type from int to string
L0rdLizard Apr 25, 2024
70d0b55
Merge branch 'base/L3-Rabbit' into fix/timeouts-merge
potat-dev Apr 25, 2024
ea1e944
Merge pull request from fix/timeouts-merge
potat-dev Apr 25, 2024
985832d
Client and Worker Registration implemented
potat-dev May 22, 2024
6ddb3b8
Optimized code 2.824229407 E+456568 times
L0rdLizard May 22, 2024
12fa99e
Merge remote-tracking branch 'origin/base/L3-Rabbit' into base/L3-Rabbit
L0rdLizard May 22, 2024
4d625c3
Small struct and cmake fix
potat-dev May 22, 2024
c3fc818
Minor server optimisation
potat-dev May 22, 2024
4cc5cd9
More robust server logs
potat-dev May 22, 2024
2a12e79
Client ready
potat-dev May 22, 2024
a3e8be4
fix port
L0rdLizard May 22, 2024
ded5762
Fix many connection errors
potat-dev May 22, 2024
f5ce62d
Client connection fix
potat-dev May 23, 2024
abecc6e
Great Server fix after 4 hours of sleep
potat-dev May 23, 2024
2bc77ba
Finally, client - server - worker connection is working
potat-dev May 23, 2024
db266b7
Parallelization to multiple wokers works
potat-dev May 23, 2024
f65842c
File logging and some test changes
potat-dev May 23, 2024
1b72742
Merge successful, good luck!
Sergeydigl3 May 23, 2024
73b986d
fix server client worker
L0rdLizard May 23, 2024
dd84176
Web UI
Sergeydigl3 May 24, 2024
8a538f5
Toggle for logs and minor worker messages fix
potat-dev May 24, 2024
33ce6df
create matrixMultiplication
L0rdLizard May 24, 2024
2bbb8e8
add matrixMultiplication to struct
L0rdLizard May 24, 2024
77df178
Matrix Multiplication task creation on Client and small log fixes
potat-dev May 24, 2024
8dc934e
Add threading option to client input
potat-dev May 24, 2024
e445ef0
Remove defines
potat-dev May 24, 2024
912beba
Remove defines - part 2
potat-dev May 24, 2024
aec0f99
Fix missing worker id update in RabbitServer::checkTaskQueue
potat-dev May 24, 2024
d09e28c
Minor fix in debug web ui
potat-dev May 24, 2024
d9db819
FIX: server crash
Sergeydigl3 May 24, 2024
e2df5a8
Add checkTaskQueue in processWorker on worker connection
potat-dev May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .run/CLIENT rabbitClientL3cmd.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="CLIENT rabbitClientL3cmd" type="CMakeRunConfiguration" factoryName="Application" PROGRAM_PARAMS="-i 2 -h localhost -p 3030" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="rabbit" TARGET_NAME="rabbitClientL3cmd" CONFIG_NAME="Debug" RUN_TARGET_PROJECT_NAME="rabbit" RUN_TARGET_NAME="rabbitClientL3cmd">
<method v="2">
<option name="com.jetbrains.cidr.execution.CidrBuildBeforeRunTaskProvider$BuildBeforeRunTask" enabled="true" />
</method>
</configuration>
</component>
7 changes: 7 additions & 0 deletions .run/SERVER rabbitServerL3cmd.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="SERVER rabbitServerL3cmd" type="CMakeRunConfiguration" factoryName="Application" PROGRAM_PARAMS="-p 3030" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="rabbit" TARGET_NAME="rabbitServerL3cmd" CONFIG_NAME="Debug" RUN_TARGET_PROJECT_NAME="rabbit" RUN_TARGET_NAME="rabbitServerL3cmd">
<method v="2">
<option name="com.jetbrains.cidr.execution.CidrBuildBeforeRunTaskProvider$BuildBeforeRunTask" enabled="true" />
</method>
</configuration>
</component>
7 changes: 7 additions & 0 deletions .run/WORKER1 rabbitWorkerL3cmd.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="WORKER1 rabbitWorkerL3cmd" type="CMakeRunConfiguration" factoryName="Application" PROGRAM_PARAMS="-i 1 -h localhost -p 3030 -c 4" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" PASS_PARENT_ENVS_2="true" PROJECT_NAME="rabbit" TARGET_NAME="rabbitWorkerL3cmd" CONFIG_NAME="Debug" RUN_TARGET_PROJECT_NAME="rabbit" RUN_TARGET_NAME="rabbitWorkerL3cmd">
<method v="2">
<option name="com.jetbrains.cidr.execution.CidrBuildBeforeRunTaskProvider$BuildBeforeRunTask" enabled="true" />
</method>
</configuration>
</component>
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ CPMAddPackage(

find_package(SqliteOrm REQUIRED)
find_package(argparse REQUIRED)
find_package(nlohmann_json REQUIRED)

# Include directories
include_directories(src)
Expand Down
10 changes: 10 additions & 0 deletions INTERNAL_API.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Описание внутреннего взаимодействия API

API передает сообщение используя JSON формат.
Каждое сообщение содержит следующие поля:


## Описание примеров взаимодействия

### Воркер

1 change: 1 addition & 0 deletions conandata.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@
# To keep your changes, remove these comment lines, but the plugin won't be able to modify your requirements

requirements:
- "nlohmann_json/3.11.3"
- "argparse/3.0"
- "sqlite_orm/1.8.2"
17 changes: 14 additions & 3 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,33 @@

# Add subdirectory for protocol
add_subdirectory(protocol)

include_directories(DataModel)
# Source files
set(SOURCE_FILES
server/STIPServer.cpp
client/STIPClient.cpp
services/TaskService/TaskService.cpp
services/UserDBService/UserDBService.cpp
services/TaskQueue/TaskQueue.h
)

# Function to create executable targets
function(add_rabbit_executable target_name)
add_executable(${target_name} ${ARGN} ${SOURCE_FILES})
target_compile_features(${target_name} PRIVATE cxx_std_17)
target_link_libraries(${target_name} PUBLIC STIPProtocol argparse::argparse Boost::asio ws2_32)
target_link_libraries(${target_name} PUBLIC
STIPProtocol
argparse::argparse
Boost::asio
ws2_32
nlohmann_json::nlohmann_json
)
endfunction()

# Define executables
add_rabbit_executable(rabbitServerL2Example ${CMAKE_CURRENT_SOURCE_DIR}/../examples/server.cpp)
add_rabbit_executable(rabbitClientL2Example ${CMAKE_CURRENT_SOURCE_DIR}/../examples/client.cpp)

add_rabbit_executable(rabbitServerL3cmd cmd/serverMain.cpp rabbitCore/server/RabbitServer.cpp)
target_link_libraries(rabbitServerL3cmd PRIVATE sqlite_orm::sqlite_orm)
add_rabbit_executable(rabbitClientL3cmd cmd/clientMain.cpp rabbitCore/client/RabbitClient.cpp)
add_rabbit_executable(rabbitWorkerL3cmd cmd/workerMain.cpp rabbitCore/worker/RabbitWorker.cpp)
27 changes: 27 additions & 0 deletions src/DataModel/Client.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
//
// Created by Potato on 12.04.24.
//

#ifndef RABBIT_USER_H
#define RABBIT_USER_H

#include <string>
#include <nlohmann/json.hpp>
#include "protocol/Connection.h"

using json = nlohmann::json;

struct Client {
std::string id;
STIP::Connection *connection;
};

inline void to_json(json &j, const Client &c) {
j = json{{"id", c.id}};
}

inline void from_json(const json &j, Client &c) {
j.at("id").get_to(c.id);
}

#endif //RABBIT_USER_H
48 changes: 48 additions & 0 deletions src/DataModel/Message.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
//
// Created by Potato on 12.04.24.
//

#ifndef RABBIT_MESSAGE_H
#define RABBIT_MESSAGE_H

#include <string>
#include <nlohmann/json.hpp>

using json = nlohmann::json;

// TODO: обернуть все нижестоящие объекты в этот высооуровневый класс

// example enum type declaration
enum MessageType {
RegisterClient,
RegisterWorker,
TaskRequest,
TaskResult,
Invalid = -1
};

// map TaskState values to JSON as strings
NLOHMANN_JSON_SERIALIZE_ENUM(MessageType, {
{ RegisterClient, "registerClient" },
{ RegisterWorker, "registerWorker" },
{ TaskRequest, "request" },
{ TaskResult, "result" },
{ Invalid, nullptr },
})

struct Message {
MessageType action;
std::string data;
};

inline void to_json(nlohmann::json &j, const Message &m) {
j = json{{"action", m.action},
{"data", m.data}};
}

inline void from_json(const nlohmann::json &j, Message &m) {
j.at("action").get_to(m.action);
j.at("data").get_to(m.data);
}

#endif //RABBIT_MESSAGE_H
64 changes: 64 additions & 0 deletions src/DataModel/Task.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
//
// Created by Serge on 26.03.2024.
//

#ifndef RABBIT_TASKS_H
#define RABBIT_TASKS_H

#include <string>
#include <nlohmann/json.hpp>

using json = nlohmann::json;

// example enum type declaration
enum TaskStatus {
Created,
Queued,
SentToWorker,
Ready,
Failed,
};

// map TaskState values to JSON as strings
NLOHMANN_JSON_SERIALIZE_ENUM(TaskStatus, {
{ Created, "created" },
{ Queued, "queued" },
{ SentToWorker, "sentToWorker" },
{ Ready, "ready" },
{ Failed, "failed" },
})

struct Task {
std::string id;
std::string func;
std::string input;
std::string output;
int cores;
TaskStatus status;
std::string worker_hash_id;
std::string client_hash_id;
};

inline void to_json(json &j, const Task &t) {
j = json{{"id", t.id},
{"func", t.func},
{"input", t.input},
{"output", t.output},
{"cores", t.cores},
{"status", t.status},
{"worker_hash_id", t.worker_hash_id},
{"client_hash_id", t.client_hash_id}};
}

inline void from_json(const json &j, Task &t) {
j.at("id").get_to(t.id);
j.at("func").get_to(t.func);
j.at("input").get_to(t.input);
j.at("output").get_to(t.output);
j.at("cores").get_to(t.cores);
j.at("status").get_to(t.status);
j.at("worker_hash_id").get_to(t.worker_hash_id);
j.at("client_hash_id").get_to(t.client_hash_id);
}

#endif //RABBIT_TASKS_H
34 changes: 34 additions & 0 deletions src/DataModel/TaskRequest.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
//
// Created by Potato on 11.04.24.
//

#ifndef RABBIT_TASKREQUEST_H
#define RABBIT_TASKREQUEST_H

#include <string>
#include <nlohmann/json.hpp>

using json = nlohmann::json;

struct TaskRequest {
std::string id;
std::string func;
std::string data;
int cores;
};

inline void to_json(json &j, const struct TaskRequest &tr) {
j = json{{"id", tr.id},
{"func", tr.func},
{"data", tr.data},
{"cores", tr.cores}};
}

inline void from_json(const json &j, struct TaskRequest &tr) {
j.at("id").get_to(tr.id);
j.at("func").get_to(tr.func);
j.at("data").get_to(tr.data);
j.at("cores").get_to(tr.cores);
}

#endif //RABBIT_TASKREQUEST_H
31 changes: 31 additions & 0 deletions src/DataModel/TaskResult.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
//
// Created by Potato on 11.04.24.
//

#ifndef RABBIT_TASKRESULT_H
#define RABBIT_TASKRESULT_H

#include <string>
#include <nlohmann/json.hpp>

using json = nlohmann::json;

struct TaskResult {
std::string id;
std::string data;
int status;
};

inline void to_json(json &j, const struct TaskResult &tr) {
j = json{{"id", tr.id},
{"data", tr.data},
{"status", tr.status}};
}

inline void from_json(const json &j, struct TaskResult &tr) {
j.at("id").get_to(tr.id);
j.at("data").get_to(tr.data);
j.at("status").get_to(tr.status);
}

#endif //RABBIT_TASKRESULT_H
21 changes: 0 additions & 21 deletions src/DataModel/Tasks.h

This file was deleted.

33 changes: 33 additions & 0 deletions src/DataModel/Worker.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
//
// Created by Potato on 12.04.24.
//

#ifndef RABBIT_WORKER_H
#define RABBIT_WORKER_H

#include <string>
#include <nlohmann/json.hpp>
#include "protocol/Connection.h"

using json = nlohmann::json;

struct Worker {
std::string id;
int cores;
int usedCores;
STIP::Connection *connection;
};

inline void to_json(json &j, const Worker &w) {
j = json{{"id", w.id},
{"cores", w.cores},
{"usedCores", w.usedCores}};
}

inline void from_json(const json &j, Worker &w) {
j.at("id").get_to(w.id);
j.at("cores").get_to(w.cores);
j.at("usedCores").get_to(w.usedCores);
}

#endif //RABBIT_WORKER_H
1 change: 0 additions & 1 deletion src/client/STIPClient.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ namespace STIP {
this->connectionManager = new ConnectionManager(socket);
}


void STIPClient::receiveProcess() {
std::cout << "Start listen" << std::endl;
while (isRunning) {
Expand Down
1 change: 0 additions & 1 deletion src/client/STIPClient.h
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ namespace STIP {

void receiveProcess();
};

}

#endif //RABBIT_STIPCLIENT_H
Loading
Loading