Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace ACE with native IO implementation #2740

Draft
wants to merge 254 commits into
base: development
Choose a base branch
from
Draft

Replace ACE with native IO implementation #2740

wants to merge 254 commits into from

Conversation

0blu
Copy link
Collaborator

@0blu 0blu commented Aug 29, 2024

🍰 Pullrequest

This pr will replace ACE with a self-made but highly performant native solution.

The implementation was inspired by boost asio, meaning that everything is asynchronous and requires a callback.
Users have to guarantee the lifetime of the socket throughout the transaction (aka. use std::shared_ptr<>).

Since cMaNGOS and TrinityCore already switched to Boost and this native IO implementation is very similar to Boost, some similarities in the network code can be found and compared against.

Native system function are used. There are slightly different backend implementations on each OS:

  • Windows IOCP
  • Linux epoll (with the possibility to add io_uring support)
  • macOS kqueue

Want to test it? 🧪

Just merge these two commits into your repo.
https://github.com/0blu/vmangos-core/tree/native-squash-merge-2024-11-17
As soon this native branched is squashed-merged with development you can easily rebase->drop->cherry-pick it with the new official commit.

Todo / Checklist

  • Test on low pop server
  • Test on medium pop server (3k-6k)
  • Update build guide

0blu and others added 30 commits June 23, 2024 02:32
…ode (#2680)

* Make m_ServiceStatus volatile

* Add type size to eAuthCmd  and AuthResult

* NativeIO: Realmd: First impl of `LogonChallenge` just like cMangos

Currently only windows support. Using legacy `::select` function. Must use IOCTL in the future.

* Add "Network" to log type string

* Make m_ServiceStatus volatile (again)

* SRP6 add const to parameter

* Database add DbExecMode to force sync statements (prevent race condition)

* Reimplement most AuthSocket handers async (like cMangos)

* Use high performance winsock2 async sockets

The implementation is inspired by Boost::ASIO and Nginx

* Async networking: Split declaration and definition inside header

* Async networking: Move implementation into own folder
* Make m_ServiceStatus volatile

* Add type size to eAuthCmd  and AuthResult

* NativeIO: Realmd: First impl of `LogonChallenge` just like cMangos

Currently only windows support. Using legacy `::select` function. Must use IOCTL in the future.

* Add "Network" to log type string

* Make m_ServiceStatus volatile (again)

* SRP6 add const to parameter

* Database add DbExecMode to force sync statements (prevent race condition)

* Reimplement most AuthSocket handers async (like cMangos)

* Use high performance winsock2 async sockets

The implementation is inspired by Boost::ASIO and Nginx

* Async networking: Split declaration and definition inside header

* Async networking: Move implementation into own folder

* Fix ATTR_PRINTF on PExecute with DbExecMode

* Add function IO::Multithreading::CreateThread with nameable threads

* Add IO::Timer::AsyncSystemTimer

* Change RunEventLoop error handling

* Remove Timer was already removed error log

* Add comment explaining why we cant use `SetThreadDescription`

* Remove `MaNGOS::` prefix from `IO::` namespace
* Make m_ServiceStatus volatile

* Add type size to eAuthCmd  and AuthResult

* NativeIO: Realmd: First impl of `LogonChallenge` just like cMangos

Currently only windows support. Using legacy `::select` function. Must use IOCTL in the future.

* Add "Network" to log type string

* Make m_ServiceStatus volatile (again)

* SRP6 add const to parameter

* Database add DbExecMode to force sync statements (prevent race condition)

* Reimplement most AuthSocket handers async (like cMangos)

* Use high performance winsock2 async sockets

The implementation is inspired by Boost::ASIO and Nginx

* Async networking: Split declaration and definition inside header

* Async networking: Move implementation into own folder

* Fix ATTR_PRINTF on PExecute with DbExecMode

* Add function IO::Multithreading::CreateThread with nameable threads

* Add IO::Timer::AsyncSystemTimer

* Change RunEventLoop error handling

* Remove Timer was already removed error log

* Add comment explaining why we cant use `SetThreadDescription`

* Remove `MaNGOS::` prefix from `IO::` namespace

* Add TrinityCore's EnumFlag implementation

Reference: https://github.com/TrinityCore/TrinityCore/blob/a0e5f9db7e3fdf70ec70533e64e5653f04eeac16/src/common/Utilities/EnumFlag.h

* Fix windows #include

* Please split `socket->Write(...)` calls

* Async socket add Write for `uint8[]`

* Update AsyncServerListener.h

* Update AuthCodes.h

* Add namespace `IO::Filesystem`

* ByteBuffer add `append(std::vector<uint8> const& src)`

* Realmd: Remove old PatchHandler

* Add new `ClientPatchCache`

* Realmd: Add support for `CMD_XFER_ACCEPT`

* Remove ACE_TYPES from `XFER_DATA_CHUNK`

* Add `IO::Networking::` `IpAddress` and `IpEndpoint` structures

* Add `IO::Networking::IsInSameSubnet`

* Add function `AsyncSocket<T>::GetRemoteEndpoint()` and `GetRemoteIpString()`

* Make ReamList use new `IO::Networking::IpAddress`

* Fix `IO::Filesystem::GetAllFilesInFolder` comment
* Create NativeAliases.h

* IO::Networking moved some files around

* `IocpOperationTask` as member of `AsyncSocket` to avoid `new` memory allocation

* Native IO: Able to compile empty impl on linux

* Fix windows build

* Native IO: Linux add fs support

* Native IO: Linux, able to connect

* Fix uninitialized `m_sessionDurationTimeout`

* Fix SO_REUSEADDR on linux

* Sortof impl of unix epoll AsyncSocket

* Add own implementation of `AsyncSystemTimer`

* Code Style

* Native IO: Add all other Write handlers

* NativeIO: Linux add EPOLLRDHUP for disconnect requests

* Fix windows build

* NativeIO: Windows Code Style

* Fix EnumFlags

* Update comment
* Native mangosd: Able to "join" the game

* Add ring buffer for packets. Encryption is done in IO thread

* Rename all threads

* Add Reactor style native IO like Boost or ACE

* Add a WorldSocketMgt just like `TrinityCore`

* Readd realmd support with new IoContext stuff

* New IO ctx now compiles under linux (still some runtime bugs)

* Native: Linux use eventfd for context switch notify

* Fix windows build

* Fix usage of `std::shared_ptr<WorldSocket>`

* Make m_sendQueueIsRunning atomic

* Fix duplicated line

* Fix duplicated maiev scan in debug (#2706)

* Remove lock free packet send queue for now

* Fix linux build

* Initialize some variables at `WorldSession` to make clang-tidy happy

* NativeIO: Use std::atomic to be lock free

* Native: Also make windows IO lock free

* Fix sql field index comment

* Remove unnecessary includes

* Update mangosd.conf.dist.in

* Minor changes to native IO code and comments

* Fix initial macos bugs

* Fix some comments

* Inline MANGOS_ASSERT condition

* Add JetBrain IDE code style defaults (#2709)

* Native IO: Fix macOS build

* Reformat `mangos/Master.cpp` and `realmd/Main.cpp`

* Fix incorrect client ip realmd -> mangosd check

* Re-add `HandlePing` support for older clients

* Reformat `WorldSocket.cpp`

* Native: World/ioCtx shutdown fix

* AsyncSystemTimer: Fix potential notify loss
* NativeIO: Remove unused parts of the code

* WorldSession do `FinalizeSession()` on socket

* Rename variable

* Native IO: Don't get the filePath from a handle

* Reformat AsyncIoOperation.h

* Use inet_ntop instead inet_ntoa for thread-safety

* Fix visibility in AsyncSocket

* Clear CONTEXT_PRESENT when StopPendingTransactions
* Remove `::inet_ntop` dependency on windows

We cant use ::inet_ntoa(...) because it's not thread safe.
We cant use ::inet_ntop(...) because it's not WinXP compatible.
So we have to do it ourselves.

* Fix Windows `AsyncSystemTimer` on 32bit

* Fix `EnumFlags.h` on VS14

* Add comment to `GetRemoteIpString`

* Add max size to `m_sendQueue`
If for example a full windows path would be provided in one of the variables, it would fail
Native Branch: Replace ACE/get_opt with custom ArgparseForServer
@mserajnik
Copy link
Contributor

mserajnik commented Nov 4, 2024

@0blu

Build fails under Ubuntu/GCC with 8a78b31 when ENABLE_MAILSENDER is set:

/core/src/realmd/AuthSocket.cpp: In member function 'void AuthSocket::_HandleLogonProof__PostRecv(const std::shared_ptr<const sAuthLogonProof_C>&, const std::shared_ptr<const PINData>&)':
/core/src/realmd/AuthSocket.cpp:714:56: error: 'GetRemoteIpAddress' was not declared in this scope; did you mean 'GetRemoteIpString'?
  714 |                 mail->substitution("%originating_ip%", GetRemoteIpAddress());
      |                                                        ^~~~~~~~~~~~~~~~~~
      |                                                        GetRemoteIpString

Maybe it would be a good idea to enable all the optional features (like ENABLE_MAILSENDER) for the "test" workflows to catch stuff like this easier? Saw that that's implemented now (👍) and that the workflow currently fails with the same error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CPP A issue / PR which references CPP code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants