Build CoreCLR for Android #4296
Comments
|
count me in i will give it more time once i find it. But we will need some experience in arm assembly and c++ for tackling the very first compiling issues. |
|
Do we need to wait for LLILC to mature? I suspect that will be the key piece of the Android port. And, while LLILC JIT support would technically work on Android, an AOT compiler with aggressive dead code elimination would be required for any real applications to use CoreCLR on android. |
|
@JeroMiya A JIT will definitely be needed for a full .NET implementation, but for the moment we're trying to get started. The first thing we need, before anyone even looks at LLILC, is a working CoreCLR. As for an AOT compiler, this is just an idea, but it might be possible to get a lot of that for free by borrowing some of Google's Dalvik-to-OAT compiler in ART. It's an aggressive AOT compiler designed specifically for converting managed code on a very similar model to CIL into optimized native builds for Android devices, so it's definitely worth looking at. |
|
I think Dalvik-to-OAT is a dead-end, no offense. LLILC as an AOT would also be an aggressive AOT, and could be used for both Android, iOS, and even desktop apps on Mac/Linux/Windows. Possibly on the web via emscripten/asm.js too. An optimized AOT compiler is not a trivial project, so we should consolidate effort on one cross-platform solution. Right now LLILC is the only tech on the horizon, so I think it fits the bill both for cross-platform JIT and AOT. |
Is it even possible to have "one cross-platform solution" for something as fundamentally platform-dependent as a native code generator? |
|
Yes. LLVM, which LLILC is based on, generates native code for many platforms, including android, iOS, linux, mac, the web, etc... |
|
@masonwheeler take a look at dotnet/llilc#627 & dotnet/llilc#626. |
|
@jasonwilliams200OK OK, I'm looking at them, but I must be missing something; I don't see how either one pertains to this discussion. |
|
That was directed to "one cross-platform solution". LLILC is the answer and those ideas I pointed are to enhance its capability even further. |
|
Question out of curiosity, does CoreCLR have a portable interpreter mode, for platforms that can't JIT? |
|
Just wanted to express support and interest in this initiative... Is there an ongoing project or place where people are working towards 100% free .net on Android? Is this discussion 'it'? I'm more than willing to put all of my free time towards destroying Xamarin's livelihood, so if there's no project to join, I'll happily start one. Question: Since Microsoft has deals with Xamarin, and Xamarin operates like a patent troll, is there any chance that this effort could be undermined by their relationship? How do we make sure CoreCLR on Android doesn't get shut down? |
|
We're not trying to be malicious here, @kiwibonga, even towards companies we perceive as bad actors. I just want to see this happen in order to be able to create something for Android that is not currently possible to create due to the way Xamarin's licensing works. There's no need to worry about the project "being shut down." If you look at the official CoreCLR license file, you'll see that it's the MIT license, a standard open source license. This creates a promise that the copyright holder cannot take back. (Not a lawyer, but there's precedent. Look up the history of the Firebird database sometime.) Even if Microsoft decided not to support the Android port, there's not much they can do to keep people from working on it independently. If you want to contribute, start by forking the repository. Right now the first priority is to get CoreCLR to compile for Android/ARM32. If you can manage to do that, or to make progress towards it, please do so and submit a pull request. :) |
|
I agree with @masonwheeler - not trying to be malicious. Many higher-end app developers really need a paid product that is reliable and simple to use as a "whole package", which comes with paid support. That's an extremely important part of the .Net ecosystem. I also don't think Microsoft is explicitly against an iOS/Android port. It just isn't one of their priorities right now - they have enough work getting CoreCLR ported to Mac/Linux. While it is expensive, Xamarin's product is a viable solution for some, so it makes sense that iOS/Android support hasn't been a Microsoft priority yet. If there's one thing I've learned in this industry, it's to never assume ill intent. That being said, I do believe the .net ecosystem as a whole is severely held back in terms of broad adoption by the lack of a portable client AOT runtime that is free and permissively licensed and runs on Android/iOS. The general industry trend on the client-side from open platforms to closed walled gardens has rendered Mono's LGPL license unusable for most modern client-side development (iOS/Android/consoles/Mac-app-store, or any other app store model), so this is a pretty large hole in the ecosystem. There are significantly more developers priced out of Xamarin than can afford it, and ecosystem adoption is judged by number of developers, not how much revenue each developer makes on their product. Now, back to the original issue at hand: I think Android support should be based on LLILC, and progress in three stages: Stage 1: LLILC as a JIT engine. LLILC only has support for using LLVM as a JIT engine, and Android (but not iOS), supports JIT. Stage 1 would only be usable for development and experimentation - it's not a viable solution for a production app due to the app size cost, startup time cost, and runtime cost of JIT. Stage 2: LLILC as an AOT compiler. LLILC's AOT support is a ways off, but once implemented would enable practical use of CoreCLR in a production Android app, assuming platform bindings were available (platform bindings are beyond the scope of this issue). Developers may continue to use the stage 1 JIT mode to reduce the build time during incremental development. Stage 3: Optimization. Depending on LLILC's AOT implementation, we may need to do additional dead-code elimination to get the deployed application size down to reasonable levels. This is also where we might need to make Android-specific optimizations to the runtime. |
|
@JeroMiya LLILC is definitely an important step, but it's completely distinct from this one. LLVM already supports an ARM compilation target, so a lot of the hard work is already done there. LLILC is about translating IL generated from C# (or other CLR languages) to native code for execution, but even with that, we still need the runtime, a good deal of which is not written in C#. Until we have a working CoreCLR, LLILC for Android is useless. |
|
@masonwheeler porting of the runtime to Android is implied in stage 1. This implies also any platform layer implementation needed for Android, e.g. native threading, file streams, and other quirks of the android platform, relative to the Linux port of CoreCLR. |
|
Guys, lets not forget that the products from Xamarin are based on open source software; the only real difference between them is the software licencing involved. Not everyone feels that a license that allows what the MIT license does is one they want to contribute to, just like some folks contributing to this won't contribute to one with a GNU license. |
|
What's your point, @n8ohu? |
|
should we actually do something practically guys ? a try can not hurt anyone so if someone can contribute something they should start right now. |
|
@shahid-pk Are you still actively on this? I saw @masonwheeler's comment about what you have so far, but it's not clear where or why you stopped -- some documentation could help. I've been looking at the build.cmd script because I work on Windows; I'll probably be going through the same motions you did to add an "arm" argument, and to compile with clang/gcc... Will definitely have more time to work on this in the coming week. |
|
@kiwibonga initially i stopped because i got into the compiling stage where i was hitting amd64 assembly code (The PAL layer contains a lot of assembly/amd64 assembly). And i am very bad at assembly code so i was looking for help. Then i toasted my linux hard drive and now i only have windows i will have linux again soon but i am afraid i can not help in low level c++ and assembly. I will provide a wiki in my branch about what i have done and what needs to be done soon. |
|
@kiwibonga My point is that Xamarin is paying for the development of Mono, which is an open source project, with the licensing fees for the commercial Xamarin products; they chose long ago to use an open source license that required certain things which some developers choose to consider restrictive. Microsoft, like Apple before it, chose a different license, with all the potential consequences that could arise from that as we have seen with relatively recent reductions in the amount of core code for Darwin (the core of Mac OS X) that used to be compilable that no longer are. Either way, there is going to be an increasing amount of software that uses Mono or CoreFX/CoreCLR that runs on Linux, so engaging in a "holy war" of sorts isn't of benefit to the community as a whole. |
|
As i have stated elsewhere too i must say i have nothing personally against anyone or any company. Xamarin must be good for some and bad for some i do not care we should all in fact don't need to care about that. What we should focus on right now is creating a free port of coreclr to arm. Discussing who else is doing this and who is charging for it is not relevant here. We should mind our own job. |
|
If you'd like to continue discussion of the relative merits of open source licenses as it pertains to coreclr, xamarin's business model, and so forth, may I suggest further discussion be moved to the .net foundation forums, or a social media platform of your choice? This will help the implementors keep focused on the actual task without extra noise. I will delete my own off topic commentary as soon as I'm not on a phone browser. Thanks! |
|
This is less an Android, more an ARM question generally. I've started an attempt to get PAL at least building for ARM, but my lack of knowledge regarding SEH is problematic. Can anyone point me in the right direction so that I can hope to update src/pal/src/exception/seh-unwind.cpp for ARM, or otherwise find a solution so that I might move on. Thanks. |
|
There's some documentation about SEH on MSDN: https://msdn.microsoft.com/en-us/library/ms679270(v=vs.85).aspx The code uses functions from a package called libunwind (docs: http://www.nongnu.org/libunwind/docs.html) Mono has some info on exception propagation and SEH: http://www.mono-project.com/docs/advanced/pinvoke/#runtime-exception-propagation You may be able to get some inspiration from mono's code: If you don't want to handle it and just want to compile, I think you can get away with making all the functions empty stubs with dummy return values -- it just means exceptions will be invisible to exception handlers across native/managed boundaries, which is probably non-critical at this very early stage. |
|
You don't need to understand SEH to do an ARM port here. You just need to update the architecture specific unwinding code to copy the callee-saved registers and $ip into the WinContext. NOTE: You'll also need to update pal.h to include the CONTEXT structures and various bits for arm. |
|
Before you are too much into ARM, just wanting to point out there is x86 Android too (mostly for the Intel Atom SoCs). Maybe it is easier to first bring it there first (e.g. fixing problems from library differences). Once it runs cleanly port it to the other ISA. Just saying. |
|
That was something I noted, though, Android isn't really where my interest lies, even if that is where this issue started. Currently I am attempting to get it running on Ubuntu 14.04 on ARM, rather than Android with everything that entails. |
|
This issue maybe for android but we should take it as port for arm32 generally not just android. Our testing platform maybe android. |
|
Just wanted to update. I've been focused on using the Android NDK tools to cross-compile. I've had success getting a build started with android-cmake (https://github.com/taka-no-me/android-cmake). Commands to cross compile on Windows (run from folder where android-cmake was cloned): "C:/NVPACK/android-ndk-r9d" is the NDK directory on my machine and "C:/libs/coreclr/coreclr" is where the coreclr repo is cloned. This fails to build right away, of course, but it takes care of a good chunk of the cross-compiling/platform detection work and provides useful defines you can use in CMakeLists. |
|
Update on the above two comments:
|
|
Update: |
|
Update: once dotnet/coreclr#9940 has landed, building CoreCLR + CoreFX from master should get you a version of .NET Core which you can cross-build for arm64 Android and run "Hello World" on Android. |
|
I'm really excited to see this coming to light. Awesome work! Thanks to the community (and MS) for working on this together 👍 |
|
Okay so I think dotnet/coreclr#9940 was merged so it landed, right? What happens to this? Is there any kind of documentation or remaining work regarding .NET Core on Android? Thanks! |
|
@darkguy2008 the current state is that you can cross build coreclr and corefx repos for Android ARM32 / ARM64 on Linux and use the resulting binaries for running a dotnet core application using the |
The Android RID has been added to coreclr and corefx; it needs to be added to the core-setup repo.
I've moved all development work to Ubuntu 16.06 which means termux-specific modifications have been removed. For progress...and to help out (any and all help is very welcome) please see this issue in core-setup. |
I'm sorry, for those of us not familiar with Termux, what does the one have to do with the other? |
|
Is this project dead? |
|
I don't think anyone is actively working on this. Lots of patches have landed which improved compatibility, but some of the scripts may be out of date. |
|
The android rootfs builder script is kind of out of date, the library versions are old and one of the libraries doesn't really exist anymore (the android support library- afaik). But after some editing it is possible to build for ARM64, no idea if it works on Android though as I couldn't try it out. |
|
I was able to installed an dotnet core arm64 app (PowerShell) on Android with the app UserLAnd available in the Google Playstore. https://github.com/CypherpunkArmory/UserLAnd @qmfrederik What do you think of it ? |
|
|
|
@fMichaleczek wow great progress. Please document somewhere how you did it step by step. |
For PowerShell, install a Debian and follow the instructions from here Experimental-Installing Powershell Core on Linux Raspbian. I have tested with PowerShell 6.2.1 arm64, archive links : github/PowerShell and I plan to tested PowerShell 7 Preview 1 (dotnetcore 3.0) |
|
@fMichaleczek Awesome! So you're running .NET Core on Debian on Android, correct? |
|
@qmfrederik Yes correct. A sort of WSL for Android. Path /Storage is shared with the host, the "container" is accessible through SSH, VNC and X11. I've tested with success on Android with PowerShell ARM build and Android-X86 with x64 PS build :
I don't see any issue while testing PowerShell. I download some modules from PSGallery and test the more advanced code without any error. I haven't success to setup X11 and wasn't able to test GUI like Avalonia. My next plan will be to test this document : UserLAnd-Adding an App and try to compile an APK. |
|
Any further thoughts on when this might be addressed? |
|
Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process. This process is part of our issue cleanup automation. |
dotnet-policy-service
bot
added
backlog-cleanup-candidate
and others
Now that we've got a good idea of what we need to do for Android support, let's try and make it happen!
The first step is to build CoreCLR for ARM32, which is what the majority of Android devices run on today. @shahid-pk has contributed some work towards the goal of getting it to build, but he says it's incomplete and still needs some work which he doesn't have time to finish at the moment. Can anyone review this and possibly contribute some missing pieces? I'd help but I know very little C++ and even less of shell-scripting.
The text was updated successfully, but these errors were encountered: