Syncthing: request: add darwin/arm64 build for macOS ARM

1

Apple Silicon M1 Macs are out, and a native ARM version of syncthing would be awesome. AIUI, golang already has support for darwin/arm64 (edit: accidentally wrote darwin/amd64 originally by accident) , so hopefully this isn't much work on your end. I'm happy to help with testing if you need any as I have the hardware in hand.

sneak picture sneak  ·  18 Nov 2020

Most helpful comment

2

Go supports darwin-arm64 (in the macOS/M1 sense) in the "tip" (current development) version only, not in the released 1.15. We have a preview build with this, it's available in the latest release candidate for example: https://github.com/syncthing/syncthing/releases/tag/v1.12.0-rc.3

I don't yet have an M1 Mac so this is completely untested, might not even run. There are new requirements around code signing for example that I'm not sure if we fulfill or not. Testing appreciated.

calmh picture calmh  ·  19 Nov 2020

All comments

2

Go supports darwin-arm64 (in the macOS/M1 sense) in the "tip" (current development) version only, not in the released 1.15. We have a preview build with this, it's available in the latest release candidate for example: https://github.com/syncthing/syncthing/releases/tag/v1.12.0-rc.3

I don't yet have an M1 Mac so this is completely untested, might not even run. There are new requirements around code signing for example that I'm not sure if we fulfill or not. Testing appreciated.

calmh picture calmh  ·  19 Nov 2020
0

Will test today and report back.

sneak picture sneak  ·  19 Nov 2020
0

Off Topic: just noticed the changelog, and: oh wow, untrusted encrypted devices in 1.12, that's such a great feature I've hoped for for so long! Thanks so much for all of your hard work!

sneak picture sneak  ·  19 Nov 2020
1

@calmh FYI I tried the new build and get this panic:

$ ./syncthing
panic: qtls.ClientHelloInfo doesn't match

goroutine 1 [running]:
github.com/marten-seemann/qtls-go1-15.init.0()
    github.com/marten-seemann/[email protected]/unsafe.go:20 +0x1b4
ssgelm picture ssgelm  ·  20 Nov 2020
0

Thanks for testing. Of course the quic package blows up, that will probably need to be disabled until it's updated for Go 1.16.

calmh picture calmh  ·  20 Nov 2020
0

I've been having an issue where the amd64 version of syncthing spikes to 100% cpu and hangs so I had to turn it off for now so if you'd like me to test anymore arm64 versions please let me know! 😄

ssgelm picture ssgelm  ·  20 Nov 2020
1

Nope same exact panic.

ssgelm picture ssgelm  ·  20 Nov 2020
0

That's odd because it shouldn't be building that package at all. Maybe it's included by something else. I guess I'll have to experiment when I have M1 hardware, or when Go 1.16 is released.

calmh picture calmh  ·  20 Nov 2020
0

If I have a chance I will try to see if I can get it to build and run locally.

ssgelm picture ssgelm  ·  20 Nov 2020
0

With the latest arm builds provided, I get the same error as https://github.com/syncthing/syncthing/issues/7128#issuecomment-730752533

I tried installing the latest Syncthing through MacPorts, which theoretically is a fresh compilation job. I'm not sure what happened (no output from MacPorts) but I figured after about three hours the build system was spinning, not actually doing anything useful. Not that I expect this to be fixed here, just mentioning it for awareness. I think the issue is, as already mentioned, Go's currrent lack of support for the new architecture.

The amd64 build does work fine on my M1 Mac with Rosetta. I copied the configuration directly from my old Mac. I haven't noticed any CPU spikes or hangs, but I also don't share many files or generate much activity.

homeisfar picture homeisfar  ·  20 Nov 2020
0

Apparently the zip build didn't take the extra build tags into account, but now it should so see if this helps: https://build.syncthing.net/buildConfiguration/Syncthing_BuildMac/86187?buildTab=artifacts&guest=1

I read something about having to set GODEBUG=asyncpreemptoff=1 when running, not sure if that was for amd64+Rosetta or arm64 native or not at all.

calmh picture calmh  ·  20 Nov 2020
1

The arm build you provided launches successfully (./syncthing) but with filesystem watcher errors: Error while trying to start filesystem watcher for folder "org" (uykt5-no3s9), trying again in 1m0s: watching is not supported

The amd64 build in your latest link launches with Rosetta successfully with no errors.

homeisfar picture homeisfar  ·  20 Nov 2020
0

Yeah, that's expected, no fs notifications due to cross compilation. Will need a native build for that, or more complicated build setup.

calmh picture calmh  ·  20 Nov 2020
1

Apparently the zip build didn't take the extra build tags into account, but now it should so see if this helps: https://build.syncthing.net/buildConfiguration/Syncthing_BuildMac/86187?buildTab=artifacts&guest=1

I read something about having to set GODEBUG=asyncpreemptoff=1 when running, not sure if that was for amd64+Rosetta or arm64 native or not at all.

FWIW, the amd64 version started up fine on my M1 based Mac mini, but soon after ran into the following error:

[EB7QQ] 16:29:21 INFO: Completed initial scan of sendreceive folder Xxxxx
assertion failed [abi_info.kind == AbiKind::TranslatedCode]: emulated forward to an arm pc that isn't in translated code. arm_pc=0x105bc44d8 abi_kind=6 emulation_interval=[0x105c094fc,0x105c09510) instruction_interval=[0x105c094e8, 0x105c09510) x86_rip=0x404e70d
(ThreadContextRegisterState.cpp:677 move_to_instruction_boundary)

Restarting it with GODEBUG=asyncpreemptoff=1 seemed to take care of that problem (thanks for that tip!), but file watching would still not work.

I'm now running the arm64 version with file watching disabled. Apart from:

[EB7QQ] 16:59:46 INFO: Listener for quic://0.0.0.0:22000: unknown address scheme "quic"

it seems to be working well.

lenlo picture lenlo  ·  22 Nov 2020
0

Is this fix committed anywhere? I still get the panic at bf7d03d when building with Homebrew (after patching --HEAD Homebrew/homebrew-core#65703).

Note that this is not a darwin/arm64 issue, but a Go 1.16 one. quic-go relies on internal implementation details of the standard library, so can break at every Go release.

The CPU spikes are probably golang/go#42700 and indeed can be worked around by GODEBUG=asyncpreemptoff=1.

FiloSottile picture FiloSottile  ·  26 Nov 2020
0

Ah, adding --tags noquic got me a working build.

May I suggest gating the quic files on go1.14,!go1.16 instead? This will ensure that the build works with any version automatically, and lets you opt in to building quic with a new version after you tested it.

FiloSottile picture FiloSottile  ·  26 Nov 2020
0

The noquic tag is a quick hack to deal with quic package oddities, not something we really want to perpetuate or systematise I think... That is, I think I prefer requiring an action to confirm that you’re creating a broken build (lacking quic support) rather than silently excluding quic.

calmh picture calmh  ·  26 Nov 2020
0

@FiloSottile

May I suggest gating the quic files on go1.14,!go1.17 instead? This will ensure that the build works with any version automatically, and lets you opt in to building quic with a new version after you tested it.

That sounds like a good idea. Would you mind suggesting that on https://github.com/marten-seemann/qtls-go1-15 instead? That would make a runtime panic become a build error, which is so much better. And all dependencies of quic would actually profit, instead of everyone having to work around quic badness on their own. There's a better chance that they will take it seriously coming from the go security lead (there's a bit of a track record there of ignoring people until a big player says the same thing as the people).

imsodin picture imsodin  ·  26 Nov 2020
0

Hmm, it would definitely be better to fix this upstream, but in this case it's a bit tricky. Until Go 1.16 is actually released they have two options: tag !go1.16 and just break anyone trying to build with Go tip regardless of whether it works (which is bad because it would stop you, us, and anyone from testing Syncthing with Go tip); or tag !go1.17 and then change it to !go1.16 every time I change something in crypto/tls, which can happen multiple times per cycle, and which would anyway introduce lag during which the panic would come back.

Instead, since QUIC is optional in Syncthing, you can just tag !go1.16 until Go 1.16 is finalized (say, until Go 1.16rc1) and QUIC support was tested. Builds with Go tip would always drop QUIC, but that's much better than not working at all.

FiloSottile picture FiloSottile  ·  26 Nov 2020
0

(This is way out left field and not strictly related to this issue, but I've been thinking for a while that we should re-evaluate quic as a whole. The idea was that the better NAT penetration would reduce the dependency on relays, but this is not what's happening with current usage stats reporting .33% quic4, 3.25% quic6 and 18% relay connections. Either our NAT detection is wrong/broken or the penetration isn't working for some other reason. (The quic6 connections might be non-NAT firewall penetration due to simultaneous opens.) It's not clear to me we're getting ROI on all the machinery behind the quic connections so we might want to either fix it or remove it...)

Anyway, given all that we may indeed make quic more optional in the meantime. Also making sure then to not log warnings about unknown protocol schemes, etc. when quic isn't compiled in.

(We also need to sort out arm64 builds with cgo before we can start releasing binaries, ideally without buying new builder hardware, although it might come to that...)

calmh picture calmh  ·  26 Nov 2020
0

Is there a physical cost associated with keeping it? If it helps that 3.5% of people, it's still saves some relay bandwidth.

Also, perhaps the brand new tcp punch through is such a hit nowadays that it steals quic's thunder 👯 😄

AudriusButkevicius picture AudriusButkevicius  ·  26 Nov 2020
0

The cost is discussing it in this and other issues related to the STUN stuff, and every time there's a new Go release upcoming. :)

calmh picture calmh  ·  26 Nov 2020
1

The build pain - sure I get that, and we can just put stuff behind a flag for people who want to build off tip can do that.

Same way every time there is a new version of go and quic hasn't cought up, we can release official builds without quic until we don't have to go chasing people, but I don't think we should rip it out all together.

AudriusButkevicius picture AudriusButkevicius  ·  26 Nov 2020