compilation under windows or cross-compilation

[femust]

Hey,

so I am running the script build_mingw.sh and I got the following error:

-- Could NOT find ZIP (missing: ZIP_LIBRARY ZIP_INCLUDE_DIR)

since on ubuntu I have this library, I'm not sure why it can't find it, I mean, I am assuming it's due to compilation under windows, but then the question arises how should I install this zip library under ubuntu for cross compilation?

[clemensschiffer]

Hi,

the dependencies have to be available in the same cross-compile configuration.
Have you seen this old issue, I had: #5
I had xerces and libzip build from source with the same cross-compile script.

Does that help?

[femust]

With xerces I solved the problem but libzip.... it's like building dependency to dependency to dependency and there is no end ...

[clemensschiffer]

For zlib there is a package available (at least in Ubunutu 18.04) then libzib is built on top of that:
apt install libz-mingw-w64-dev

I do have a docker file with all that somewhere, actually I meant to include that in the CI-process here....

[femust]

I guess for docker file example I will appreciate :) (but IMO after installing libz that you recommended I did step forward, but now linking problem with xerces), it's like never-ending story.

[clemensschiffer]

Found it.
Note: I had to redact some urls since I had those code repos hosted internally, so replace [[ url ]] parts with
a respective code repo.
I think one of the dependencies may not be on github but I downloaded it from sourceforge and put it in a internal git-repo, could also use a wget there.

After all that I still had to tell cmake where to find ASIO, I put:

    if(NOT ASIO_ROOT)
      set(ASIO_ROOT /usr/x86_64-w64-mingw32)
      if(ASIO_FIND_REQUIRED)
        message(FATAL_ERROR "ASIO headers could not be found.")
      else()

into cmake/FindASIO.cmake, you can also set the cmake-variable to that effect.
I copied all the dependencies into /var/deps to they could be copied with the build dcp executable to windows.
You can find all dependencies of your binary by running e.g. ldd dcpmaster.

Building can be quite frustrating, that's why docker is great: once you got it you got it.
Share here what did work for you.
Hopefully I'll get around to integrating this docker into CI-process so the lib is tested on win on every PR.

FROM ubuntu:18.04 

MAINTAINER Clemens Schiffer <[email protected]>

USER root 
RUN apt-get update; apt-get upgrade -y; 
RUN apt-get -y install  git unzip wget build-essential g++-7 cmake gcc-7 libicu-dev build-essential libzip-dev libxerces-c-dev libssl-dev
RUN apt-get -y install mingw-w64 binutils-mingw-w64 libz-mingw-w64-dev mingw-w64-tools
RUN apt-get -y install net-tools network-manager vim iputils-ping 

RUN update-alternatives --set x86_64-w64-mingw32-gcc /usr/bin/x86_64-w64-mingw32-gcc-posix
RUN update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix

# Directory to be mounted to the outside:
RUN mkdir -p /var/mybuild 
WORKDIR /var/mybuild

# build dir
RUN mkdir -p /var/build 

# DCPlib: Pull only so other can use by Toolchain, build later 
RUN git clone https://github.com/modelica/DCPLib.git /var/build/DCPLib
RUN cp /var/build/DCPLib/Toolchain-mingw-w64-ubuntu.cmake /var/build/

# Xerces: Pull and build
RUN git clone [[ Xerces-C URL ]]  /var/build/xerces
RUN sed -ri "s|Windows.h|windows.h|" /var/build/xerces/tests/src/ThreadTest/ThreadTest.cpp
RUN cd /var/build/xerces/ && \
	mkdir -p build && cd build && \
	cmake -DCMAKE_TOOLCHAIN_FILE=/var/build/Toolchain-mingw-w64-ubuntu.cmake ..  && \
	cmake --build . && \
	make install

# asio: Pull and build
RUN git clone [[ asio URL ]] /var/asio
RUN cd /var/asio/asio  && \
	chmod +x configure && \
	./configure --without-boost --prefix=/usr/x86_64-w64-mingw32 && \
	cd include && \
	make install

# Libzip: Pull and build
RUN git clone [[ libzip_1.5.2.git URL ]] /var/build/libzip
RUN sed -ri "s|Windows.h|windows.h|" /var/build/libzip/lib/zip_crypto_win.c 
RUN cd /var/build/libzip/ && \
	mkdir -p build && cd build && \
	cmake -DCMAKE_TOOLCHAIN_FILE=/var/build/Toolchain-mingw-w64-ubuntu.cmake ..  && \
	cmake --build . && \
	make install

# Copy dependencies to final: to be cp to /var/mybuild by docker-compose up
RUN mkdir -p /var/deps
RUN cp /usr/lib/gcc/x86_64-w64-mingw32/7.3-posix/libgcc_s_seh-1.dll /var/deps
RUN cp /usr/lib/gcc/x86_64-w64-mingw32/7.3-posix/libstdc++-6.dll /var/deps
RUN cp /usr/x86_64-w64-mingw32/lib/libwinpthread-1.dll /var/deps
RUN cp /usr/x86_64-w64-mingw32/bin/libxerces-c.dll /var/deps

# build my copy of the dcplib:
RUN cd /var/build/DCPLib && git pull
RUN cd /var/build/DCPLib && mkdir -p build && cd build && \
	cmake -DCMAKE_TOOLCHAIN_FILE=../Toolchain-mingw-w64-ubuntu.cmake .. && \
	grep -rl 'isystem' | xargs sed -i 's/isystem/I/g' && \
	cmake --build . --target install

# DCP Examples cross-compile build can start: ....

[masoud-najafi]

Hi,
I would like to build DCPLIB on Win64, but it is not clear in the Readme, if I can install DCPLIB directly in Windows, or I need to do cross-compilation with MinGW. Do I need to install Ubuntu? I wonder if there is any step-by-step help for Windows?
Thanks
Masoud
Altair

[clemensschiffer]

Cross-Compiling is easiest because you can rely on the system packages for dependencies, all the steps should be in the docker file excerpt above.

Building natively on Windows:
Basically you need cmake and a C++11 compiler such as visual compiler.
I had to build zlib, libz and Xerces-C from source and put the asio headers 2 folders up from the project.
Make sure to always select the same architecture such as -A x64.
(Note: Additionally, I had two versions of Xerces installed which confused cmake, so watch out for that.)

[femust]

@clemensschiffer
So I have a co-simulation with a few FMUs and for some reason, the results are strongly dependent on the OS that I am running, their configuration
Have a look, here is a plot of one variable from the model (orange - ground truth, blue - dcplib)

• so when I run everything (slaves and master) on Linux here is the plot
  

• when I run all slaves on Linux and master on Windows, the plot is the same
  

BUT

• when I run slaves on Windows I got the following results (don't mind the length of the plot because sim time is different)
  
• and finally, when master is on Linux and all slaves are on Windows I got the following errors:
  

do you know what may be the reason?

[clemensschiffer]

I think we should keep this issue strictly to compilation.
If you think this may be an issue with DCPLib it would be most helpful to have a self-contained minimal example that reproduces this behavior, best in a new issue.

For your example:
There should be no difference at all. Is data transferred via the master? If the master is only configuring that should make no difference where it is running (even so it should only be latency ).

Are you sure this is the exact same code including configuration, time_resolution, input/output connections and the model itself?
Are you running SRT or NRT? In NRT there should be deterministically zero difference.
If SRT do you use STC_run with time = 0, i.e. now, or STC_run with starting time, if the later is the systemtime on all systems the same?

Do the supplied examples differ per os? I can't believe that they do....

[svanimisetti]

@clemensschiffer - Thanks for posting the Dockerfile. Is it possible to build the project with cmake+MSVC generators? Based on what I read from this thread, there are other dependencies for this project (ASIO, zip, etc) - and it would be impractical (and perhaps impossible) to build this on Window using cmake+MSVC. Is the option possible option is to cross-compile on Linux using mingw-win32 or mingw-win64?

[femust]

@svanimisetti from my experience use pacman.
In powershell

iex ((New-Object System.Net.WebClient).DownloadString('[https://community.chocolatey.org/install.ps1')](https://community.chocolatey.org/install.ps1%27))

choco install mingw -y

choco install cmake --installargs '"ADD_CMAKE_TO_PATH=System"' -y

install msys2 https://www.msys2.org/

and in msys

pacman -Syu
pacman -Su
pacman -S --needed base-devel mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-zlib
pacman -S mingw-w64-x86_64-libzip
pacman -S mingw-w64-x86_64-boost
pacman -S mingw-w64-x86_64-xerces-c
pacman -S make
pacman -S mingw-w64-x86_64-asio
pacman -S mingw-w64-x86_64-cmake

then in dcplib create a build directory and

cmake -G "MinGW Makefiles" ..
cmake --build . --target install

[clemensschiffer]

Hello @svanimisetti,
sorry for the late reply.
Basically you can do what @femust suggested, I think this still qualifies as cross-compiling.

Building natively with MSVC is possible, as I wrote above:

Building natively on Windows: Basically you need cmake and a C++11 compiler such as visual compiler. I had to build zlib, libz and Xerces-C from source and put the asio headers 2 folders up from the project. Make sure to always select the same architecture such as -A x64. (Note: Additionally, I had two versions of Xerces installed which confused cmake, so watch out for that.)

You definitely need Xerces-c follow the instructions here:
https://xerces.apache.org/xerces-c/build-3.html
It is also a cmake project.
ASIO(the non-boost variant) is header only, so no compiling needed.
Libz and zlib can be omitted, if you do not need the functionality.
Have you tried? Did you run into specific problems?

I hope I find the time soon to put this native build into the CI/CD process here, such that it later on it will only be copy/paste. ;)

Kind regards,
Clemens

[svanimisetti]

@clemensschiffer - Sorry for such a delayed response, but I was able to push through during the summer vacation and managed to compile DCPLib on Windows. As @masoud-najafi had also requested, I wanted to build directly on Windows with MinGW GCC, without the need to cross-compile via Linux or WSL. I was able to run all the cases successfully. If it is of any interest, I can write-up a small documentation and send PR your way. Let me know if that will be helpful. Otherwise, I can always document the learnings in this issue thread. Kindly advise.

[svanimisetti]

@femust - you have shown an example of FMU using DCP on win64 and linux64.

So I have a co-simulation with a few FMUs and for some reason, the results are strongly dependent on the OS that I am running, their configuration

How are you able to combine the FMU with DCP? I am trying to achieve a similar objective. I am now able to compile the DCPLib on win64 and linux64. My next step is to embed the DCPLib inside FMU and connect a co-simulation master FMU on win64 to slave on linux64. I am running into issues. Do you have any documentation demonstrating how you achieved this?

I don't mean to hijack this issue thread, but any information you can share will be greatly appreciated.

[svanimisetti]

If it is of any interest, I can write-up a small documentation and send PR your way.

@clemensschiffer - I just noticed that the documentation for installation is in the wiki (https://github.com/modelica/DCPLib.wiki.git). Unfortunately, there is no way to modify the wiki attached to main repo and send a pull request. Looks like the only way is to document the steps here.

[clemensschiffer]

Hello @svanimisetti,

thank you for your interest and the feedback!
If you needed any steps to compile natively on windows other then or more detailed then here #62 (comment), feel free to post them here, or append to the readme with a PR.
Always happy for contributions, it makes it easier for others :)

Coupling FMU<->DCP is a separate topic.
I have used the DCPLib to call a FMU and make it available as a DCP-slave, I have used the FMILibrary for this.
Of course you can also compile a FMU that then starts a DCP-slave. I think the first makes more sense as you use an existing FMU in a distributed master setting in DCP. Depends on your use case!

Kindly,
Clemens

[svanimisetti]

@clemensschiffer - Thanks for the direction. I would rather not pollute your current README. I will document the steps here for posterity and your review. If it seems acceptable to you, please merge it into the Wiki later. I will post the steps by end of this week.

[svanimisetti]

As discussed, I am documenting the steps I used to successfully compile DCPLib (along with the examples) on Windows 10 64-bit. I hope this is useful for anyone trying to compile the libraries on Windows 10.

Build Instruction for Windows 10 64-bit using MinGW

Build instructions were tested on a computer with the following configuration:

• Windows 10 64-bit Version 10.0.19044
• $ uname -a [ MINGW32_NT-6.2 xxx 1.0.19(0.48/3/2) 2016-07-13 17:45 i686 Msys ]
• $ git --version [ git version 2.37.1.windows.1 ]
• gcc -v [ gcc version 6.3.0 (x86_64-posix-seh-rev2, Built by MinGW-W64 project ]
• cmake-3.23.2 and ninja-1.11.0

NOTES:

• MSYS bin folders should be added to beginning of search path in Git-Bash so that make and other utilities from MSYS are found in-favor of those in Git-Bash
• GCC binaries were added to beginning of search path
• Ninja was installed to CMake binary folder, which was then added to beginning of search path
• All of the following configuration and build steps were performed in Git-Bash

Gather Dependencies:

cd /c/Users/<username>/Downloads
wget https://sourceforge.net/projects/asio/files/asio/1.28.0%20%28Stable%29/asio-1.28.0.zip
wget https://dlcdn.apache.org/xerces/c/3/sources/xerces-c-3.2.4.zip
wget https://libzip.org/download/libzip-1.10.0.tar.gz

Build Dependencies

When building dependencies for Windows 10, it is advisable to use --install-prefix, as without this option, the build is installed to C:\Program Files (x86) by default, if not building in the MSYS/Git-Bash prompt. For these instructions install_dir in user Downloads folder is used to collect all build artifacts. For a more permanent solution, use /usr/local as prefix in MSYS/Git-Bash prompt.

Also, since Ninja build system was used, the appropriate generator name is passed to CMake. Please refer to CMake-Ninja documentation for more details.

ASIO

cd /c/Users/<username>/Downloads
unzip asio-1.28.0.zip
cd asio-1.28.0
./configure --without-boost --prefix=/c/Users/<username>/Downloads/install_dir
cd include
make install

Xerces-c

cd /c/Users/<username>/Downloads
unzip xerces-c-3.2.4.zip
cd xerces-c-3.2.4
mkdir build
cmake -S . -B ./build -G "Ninja" --install-prefix /c/Users/<username>/Downloads/install_dir
cmake --build ./build --target install

LibZip

cd /c/Users/<username>/Downloads
tar xvzf libzip-1.10.0.tar.gz
cd libzip-1.10.0
cmake -S . -B ./build -G "Ninja" --install-prefix /c/Users/<username>/Downloads/install_dir
cmake --build ./build --target install

Build DCP Project

Build DCPLib

NOTE: In the following instructions, CMAKE_PREFIX_PATH is set to install_dir (see above) so that CMake can find the dependencies built in previous steps (asio, xerces-c, zip).

cd /c/Users/<username>/Downloads
git clone https://github.com/modelica/DCPLib.git && cd DCPLib
mkdir build
cmake -S . -B ./build -G "Ninja" --install-prefix /c/Users/<username>/Downloads/install_dir -D CMAKE_PREFIX_PATH=/c/Users/<username>/Downloads/install_dir
cmake --build ./build --target install

Build DCP Test Examples

for d in /c/Users/<username>/Downloads/DCPLib/example/spring_damper_*; do
cd $d
mkdir build
cmake -S . -B ./build -G "Ninja" --install-prefix /c/Users/<username>/Downloads/install_dir -D CMAKE_PREFIX_PATH=/c/Users/<username>/Downloads/install_dir
cmake --build ./build --target install
done

Run DCP Test Examples

NOTE: must run slaves before master - otherwise will see following error

ERROR Network problem in UDP_IPv4 driver. Error Message: No connection
could be made because the target machine actively refused it.

Open three command Git-Bash windows and set export PATH=~/Downloads/install_dir/bin:$PATH in each. Run each of the following commands from the three terminal windows.

• Git-Bash Terminal 1:

cd /c/Users/<username>/Downloads/DCPLib/example/spring_damper_slave1
./dcp_msd1_slave.exe

• Git-Bash Terminal 1:

cd /c/Users/<username>/Downloads/DCPLib/example/spring_damper_slave2
./dcp_msd2_slave.exe

• Git-Bash Terminal 1:

cd /c/Users/<username>/Downloads/DCPLib/example/spring_damper_master
./dcp_msd_master.exe

Clean-up

Simply delete install_dir from the Downloads folder once testing is done. For more permanent installation, as indicated above - use /usr/local as installation prefix in MSYS.