~eliasnaur/gio#170: 
Improvements for gogio

I really like gogio because it's easier than mess with Java directly and use gradle and such. However there's some limitations that is really bad, or it's not well documented.

Some changes includes:

  1. Check the version of javac, warning if some unsupported version is been used.

  2. Check if there's any themes.xml or themes-v21.xml in the main folder. That will make possible to change colors such as android:statusBarColor.

  3. Check if there's some manifest.xml and merge with the default one. That makes possible to change the screenSize|screenLayout|smallestScreenSize|orientation|keyboardHidden stuff, or add some permissions.


The way of "merge", means that Gio uses the default and merges with some custom themes.xml or manifest.xml. So, you can have some manifest.xml EXACTLY like:

<manifest xmlns:android="http://schemas.android.com/apk/res/android" />
        <uses-permission android:name="android.permission.BLUETOOTH"/>
        <uses-permission android:name="android.permission.CAMERA"/>
</manifest>

It's not an valid manifest, far I'm aware of. But, that will be merged with the defaults, so, that will be the result:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
        package="localhost.Kitchen"
        android:versionCode="1"
        android:versionName="1.0.1">
        <uses-sdk android:minSdkVersion="16" android:targetSdkVersion="28" />
        <uses-permission android:name="android.permission.INTERNET"/>
        <uses-feature android:glEsVersion="0x00020000" android:required="false"/>
        <uses-feature android:name="android.hardware.type.pc" android:required="false"/>
        <uses-permission android:name="android.permission.BLUETOOTH"/>
        <uses-permission android:name="android.permission.CAMERA"/>
        <application  android:label="Kitchen">
                <activity android:name="org.gioui.GioActivity"
                        android:label="Kitchen"
                        android:theme="@style/Theme.GioApp"
                        android:configChanges="screenSize|screenLayout|smallestScreenSize|orientation|keyboardHidden"
                        android:windowSoftInputMode="adjustResize">
                        <intent-filter>
                                <action android:name="android.intent.action.MAIN" />
                                <category android:name="android.intent.category.LAUNCHER" />
                        </intent-filter>
                </activity>
        </application>
</manifest>

It's not only permissions, but anything will be merged. In the case of themes it's a easier way to change colors and so on.

Status
REPORTED
Submitter
~inkeliz
Assigned to
No-one
Submitted
2 months ago
Updated
2 months ago
Labels
No labels applied.

~schnwalter 2 months ago

2 & 3 - these need to be more flexible, configurations will need to be possible for sorts of platforms. And all sorts of configurations will be required and they will change from platform to platform.

Also, ~eliasnaur, you'll need to watch out for what features are integrated into the Gio core and exposed behind a custom API or what will be exposed as "native" configuration to the consumer. You'll need to find a way to avoid the problems that the Cordova platform has ran into, where a lot of the basic features required even for simple applications are not in the core and the plugins are basically abandoned, making it hard for anyone to choose the platform. (Thanks for all the hard work.)

P.S. ~inkeliz, watch out for the content behind the status bar, it won't be visible if you set an opaque color. I would prefer to see some sort of API that will expose the height of the status bar and actions bar.

~eliasnaur 2 months ago

Hi,

It's not great to have a single issue like this be about many different things. Please create a focused issue for each feature.

That said, some comments below.

On Mon Nov 9, 2020 at 16:45, ~inkeliz wrote:

I really like gogio because it's easier than mess with Java directly and use gradle and such. However there's some limitations that is really bad, or it's not well documented.

Some changes includes:

  1. Check the version of javac, warning if some unsupported version is been used.

Indeed. I think that's a straightforward feature to implement. Would you like to prepare a patch?

  1. Check if there's any themes.xml or themes-v21.xml in the main folder. That will make possible to change colors such as android:statusBarColor.

  2. Check if there's some manifest.xml and merge with the default one. That makes possible to change the screenSize|screenLayout|smallestScreenSize|orientation|keyboardHidden stuff, or add some permissions.

I may be wrong, but merging of XML files sounds like a can of worms. I'm open to replacing AndroidManifest.xml if it exists in the source directory.

That said, I hope in future that common manifest tweaks can be achieved with source code annotations. For example, permissions are already supported through magic imports. See https://pkg.go.dev/gioui.org/app/permission.

~inkeliz 2 months ago

Abou the javac I think I can do. (:

The issue about "replacing" is because it needs some values from gogio itself. For instance, I need to access HTTP connection (for testing). That need the <application android:usesCleartextTraffic="true"> ...... </application>. But, the content inside the <application> is "unknown" (the gogio is responsable to create the <activity> with org.gioui.GioActivity, and so on).

The replacing seems easier. But, in that case, I think gogio should create the manifest file on the directory when the project lives, to be able to change that.

~thetooth 2 months ago

~eliasnaur perhaps as an alternative we add a flag like "--export" that dumps the auto generated android code/assets into a chosen folder and writes a basic make file to repeat the normal build steps. I'm not a java developer so maybe we can already just call gradle+go build from the temp folder for a complete build? If that's all that needs to be done it would give us the best of both worlds, no changes to gio core and users have complete control of android build process and can always rerun gogio and merge the gio activity with their own version if there are any major changes to it.

~schnwalter 2 months ago

~thetooth, instead of an --export flag, I would prefer to switch the build system to use an Afero based in-memory FS. In this way, various hooks can be provided to expose the FS in various moments: BeforeCompile, AfterCompile, BeforeArchive, AfterArchive.

There would be quite a few advantages to switching to an Afero based FS:

  • it would speedup the builds because the disk would no longer be used to store temporary files.
  • it would speedup testing.
  • it could also be used to provide some sort of partial build cache, to speedup subsequent builds.
  • it could also be used to implement something similar to the Hot Module Replacement from the web-world -- if there's a need for things like that.
  • the Hugo project uses a build system based on a in-memory Afero FS and it has proven to work quite nice for them. But they have a custom FS, which isn't really needed what we have in gogio.
  • because it supports various types of FS: zipFS, tarFS, etc, it could then be used to provide different build outputs for Snap packages, Makeself self-extracting archives , ISOs and what not.

And to implement these hooks, I would use something like Magefile (a go-based Makefile alternative). The mage file containing the hooks would look something like this:

// $ cat magefile.go
// build +mage

package main

func BuildStart(fs afero.Fs) fs afero.Fs, error {} // or BuildStartHook()
func BeforeCompile(fs afero.Fs) fs afero.Fs, error {}
func AfterCompile(fs afero.Fs) fs afero.Fs, error {}
func BeforeArchive(fs afero.Fs) fs afero.Fs, error {} // or BeforePackage()
func AfterArchive(fs afero.Fs) fs afero.Fs, error {} // or AfterPackage()
func BuildFinish(fs afero.Fs) fs afero.Fs, error {}

And, if needed, these hooks can be used to expose a brand new build platform that isn't supported by the Gio core team. Or internal functions could be made replaceable, a user could completely change the archiveAndroid() function. This way, the build system is very flexible and once the API is set in stone, the Gio core team doesn't need to maintain all sort of build scenarios for strange platforms that they can't test.

~eliasnaur, what do you think about this? Would you like me to look more into this and provide some patches? A first one for switching to a in-memory FS

~joe-getcouragenow 2 months ago

i use mage and afero for exactly these reasons on other golang and non golang projects and its really great. give you agnostic base and makes it much easier to onboard new devs on any os

~eliasnaur 2 months ago

mage, afero, --export...

Your suggestions sounds like gogio should compete with the build tools from the Android SDK. Granted, the Android tools are awful, but trying to replicate every existing feature and track future features is futile. And even if we succeed, it seems to be a shame to pour the work into a Gio specific tool and leave out Fyne, Gomobile, and other ways to include Go code in your app.

There's a reason the original gogio tool only did -buildmode=archive; the idea is for gogio to handle the Go and Gio specific build steps and then get out of the way. The intention for -buildmode=exe is to quickly get something running on a device.

Note that there's nothing wrong with carefully designing Gio-specific ways of manipulating AndroidManifest.xml and other metadata such as importing magic permission packages or using appicon.png from the app package. I'm only sceptical of escape hatches that don't grant you complete freedom.

So I suppose my question is: however inconvenient, what's missing from gogio -buildmode=archive and, say, a separate Gradle project? That's what I did with Tailscale[0] and it works great.

[0] https://github.com/tailscale/tailscale-android

Elias

~inkeliz 2 months ago

I never use mage/afero (...). But it seems to add some unintended complexity.

Currently, the only struggles seem to be related to manifest.xml and themes.xml. The gogio already have excellent features: it includes any .jar from any other imported library, and it already set the app's icon, and that generates the .apk, ready to run. :D

If I want some custom bar colour, there's no option. The only option is creating a custom Java with the sole propose to modify the colour of the status bar. It's doesn't seem right.

It may lack the option to provide a certificate/private-key too, but seems easy to handle.

I think gogio could read the themes.xml in the same way of appicon.png. But I think gogio should create the default themes.xml/manifest.xml in the folder, so the user can change the default (if wish to).

~eliasnaur 2 months ago

On Wed Nov 11, 2020 at 12:12, ~inkeliz wrote:

If I want some custom bar colour, there's no option. The only option is creating a custom Java with the sole propose to modify the colour of the status bar. It's doesn't seem right.

It's the merging or overriding of custom xml files that's the problem here. If you provide xml files, there no way to know what you intented to replace. Custom XML-files is really what buildmode=archive is for.

A better (and portable to iOS) approach for just the status bar color would be to introduce some mechanism to specify it. We could define that a constant with a magic name overrides the color:

package main

const statusBarColor string = "#ff0000ff"

But I think gogio should create the default themes.xml/manifest.xml in the folder, so the user can change the default (if wish to).

A feature to dump the default themes.xml/AndroidManifest.xml doesn't seem worth it. It's only slightly more inconvenient to snatch the templates from the gogio source code and fill them out.

Elias

~schnwalter 2 months ago

A better (and portable to iOS) approach for just the status bar color would be to introduce some mechanism to specify it.

I would avoid such an approach, even for simple features such as the status bar color. After that, someone will see that the splash screens needs to have special configuration for these colors. Later, someone will ask for the navigation bar color to be exposed (for android devices with no physical navigation buttons). After that, someone will want the status bar and the navigation bar to be hidden for full-screen applications (video player, games).

These are platform depended features that will be hard to maintain, especially when new platforms will come into play. It will be hard to find people that can develop and test for all platforms, for example: I don't have access to any kind of Windows or Apple device and I won't be able to help with patches that touch those platforms; but I can easily test and provide feedback any kind of Linux/BSD setup.

That's why keeping a minimal toolchain API and with some generic hooks could be one of the best solutions. It will allow people to tie into the build process and make changes at each step. This way, if a platform is not supported, it's easy for a 3rd party to add support for it. Some time ago, we used to have FirefoxOS and Ubuntu Phone, now they are gone, but we have Pinephone and Librem phones. We have Flatpak and Snap, Universal Windows Platform, the Arch AUR an many other platforms that the Gio team should not care about, but a simple "AfterBuild" hook would do wonders for the flexibility of the build process.

If the current code is cleaned up a bit (and I've already done some work on those lines), then anyone can go crazy with their own build systems. Take for example this transformation pipeline for TypeScript projects, it can be very easily adapted to output any kind of library, it was first created for Angular CLI to build Angular libraries, but many people (including me) are using that code to build libraries for various frameworks: Angular, React, Vue.

When it comes to configurations, many people will want a lot of features, and it's impossible to give them what they want. And the current archive dump it's a bit tricky to work with. The afero.FS interface is very small and only depends on the standard os package, if needed it can be replaced at any time. The same is valid for "mage", which uses the go build tags to create these kind of hooks, and if it's needed, it will be trivial to provide a minimalistic replacement.

-- Walter S.

~eliasnaur 2 months ago

On Thu Nov 12, 2020 at 1:29 AM CET, ~schnwalter wrote:

A better (and portable to iOS) approach for just the status bar color would be to introduce some mechanism to specify it.

I would avoid such an approach, even for simple features such as the status bar color. After that, someone will see that the splash screens needs to have special configuration for these colors. Later, someone will ask for the navigation bar color to be exposed (for android devices with no physical navigation buttons). After that, someone will want the status bar and the navigation bar to be hidden for full-screen applications (video player, games).

FWIW, some of these features can be exposed as runtime facilities. Your point stands for features that need to go into metadata.

That's why keeping a minimal toolchain API and with some generic hooks could be one of the best solutions. It will allow people to tie into the build process and make changes at each step. This way, if a platform is not supported, it's easy for a 3rd party to add support for it. Some time ago, we used to have FirefoxOS and Ubuntu Phone, now they are gone, but we have Pinephone and Librem phones. We have Flatpak and Snap, Universal Windows Platform, the Arch AUR an many other platforms that the Gio team should not care about, but a simple "AfterBuild" hook would do wonders for the flexibility of the build process.

Hooks still need to be maintained and kept backwards compatible.

If the current code is cleaned up a bit (and I've already done [some work on those lines](https://lists.sr.ht/~eliasnaur/gio- patches/%3C160514094789.21796.7525879750895479640-0%40git.sr.ht%3E)), then anyone can go crazy with their own build systems. Take for example this [transformation pipeline](https://github.com/ng-packagr/ng- packagr/blob/v11.0.1/docs/transformation-pipeline.md) for TypeScript projects, it can be very easily adapted to output any kind of library, it was first created for Angular CLI to build Angular libraries, but many people (including me) are using that code to build libraries for various frameworks: Angular, React, Vue.

When it comes to configurations, many people will want a lot of features, and it's impossible to give them what they want. And the current archive dump it's a bit tricky to work with. The afero.FS interface is very small and only depends on the standard os package, if needed it can be replaced at any time. The same is valid for "mage", which uses the go build tags to create these kind of hooks, and if it's needed, it will be trivial to provide a minimalistic replacement.

Simple as mage and afero may be, what's wrong with buildmode=archive? As you say, people will want all sorts of platform features, and it seems giving them maximal freedom with an .AAR file is best. I assume the themes.xml file in an AAR file can be overridden as requested; AndroidManifest.xml certainly can. And if not, we can leave out themes.xml from the AAR file, and document the suggested theme content.

Elias

~thetooth 2 months ago

what's wrong with buildmode=archive?

What's wrong for me personally, is I was unable to find documentation on using this option successfully, I actually ended up using the --work flag and building my own build script from scratch...

So perhaps all that is needed to satisfy this issue is to provide a user guide for using gio with customized ios or android targets. And ideally specify how we can use the gogio tool to generate the boiler plate to get up and running immediately with the most bare bone tools that are already part of the SDKs.

~eliasnaur 2 months ago

On Fri Nov 13, 2020 at 12:35 AM CET, ~thetooth wrote:

what's wrong with buildmode=archive?

What's wrong for me personally, is I was unable to find documentation on using this option successfully, I actually ended up using the --work flag and building my own build script from scratch...

So perhaps all that is needed to satisfy this issue is to provide a user guide for using gio with customized ios or android targets. And ideally specify how we can use the gogio tool to generate the boiler plate to get up and running immediately with the most bare bone tools that are already part of the SDKs.

That's a good idea. Would you like to contribute a first revision to https://git.sr.ht/~eliasnaur/giouiorg?

Elias

Register here or Log in to comment, or comment via email.