Pthreads

[bas]

Now that library support is in place, I've been trying out pthreads. This is quite a weird library it seems.
On OS X, compilation doesn't need a special linker flag. If you pass '-pthread', clang gives a warning about
'unused flag'. On linux, clang needs '-pthread' to build, otherwise you get an 'undefined reference to..' warning.

Nasty..

[bas]

The issue mentioned in the previous post has revealed a bigger challenge. My original idea was
to support C libraries in a similar way to Rust supports external packages (with Cargo). However,
since there is such a massive fragmentation in ways of deploying C libraries, you would either
have to depend on pkg-config or something similar to extract include-paths etc from the system
or manually keep a list per platform (which is simply undoable).

The good thing is that the requirements for C2 are much different than for a language like C++.
C++ is a systems programming language and needs extensive library support and more importantly
often use libraries installed at system level. Historically libraries were all installed at system level
and so the headers were placed somewhere in /usr/include (or similar). However, there is a recent trend
in new languages to not use system-level libraries, but just to download copies locally. This exchanges
complexity for disk-space (an easy choice). The same goes for the shift from dynamic to static libraries.
Rust (with Cargo) and LLVM/Clang are examples of this new approach.

Since C2 is meant for relatively isolated programs, I think having the user have to add a library manifest
for a specific platform is not a big problem.

This part above is only about using C-libraries from C2. There will also be a thing like C2-libraries.
For the design of this, we have a chance to shed a lot of legacy and apply 30 years of learned lessons. So
which lessons have we learned...?

[kyle]

(still catching up...)

For what it is worth, I just ported a small project from a Makefile-based build system to CMake.  The initial stages were rough.   However, it is interesting how CMake did manage to make most of the differences between building executables and libraries under Linux and Windows either disappear or at least become more minimal.

It seems like a sort of "platform shim" would be necessary as part of the port of C2 to a platform.   That would need to encapsulate the peculiarities of the platform.

As you note, other projects are just including everything they need.   While this is an easy approach to take, I have had a lot of problems with the results when a code base gets large and old.   Most of the problems I've seen with this are from Java, but I think the lessons apply here too.   The things we run into often are that there will be two, three or even four versions of Java.  Which one gets used for what task is not always obvious and sometimes not even deterministic.  Add the effort of testing for specific fixes (i.e. Java fixes for crypto problems) and it gets really fun.

One way that seems to be gaining traction is that used by Red Hat and Canonical (similar systems, but of course different implementations and syntaxes and tools.  NIH?  Never heard of it!).   This copies a bit from the Mac in that every app has its own copy of the binaries.   They did better than that by having the ability to share some of the libraries.   NixOS uses the Nix packaging system and has a very interesting method of handling dependencies.

In Nix, each package is stored in a directory with the project name and the hash (cryptographic) of the arguments used to make the project.   Thus, you get different builds with different directories when you make a library with different configurations.  Symbolic or hard links are used to mash it all together again later. 

I hope that made sense...

[bas]

I think that in general most modern solutions tend to trade complexity for disk-space. Disk space is cheap, while
complexity is growing. So, for example, instead of having a complex package system with a lot of dependency-tracking,
a (big) application would just provide all the libs it needs itself. This is also basically the idea of Docker and such.

While I agree that it is often a good trade, it also feels a bit like admitting to a loss.

Building software can be quite straightforward, as shown by many (non-mainline) languages such as Ada etc. Remember
that the C2 compiler knows everything it needs to build the C2 code in the project. The only other things it needs to understand
are libraries (basically collections of symbols with some plumbing) and linking (format depends on platform). For stuff like
opening a file, C2 re-uses llvm libraries.

While it would be very nice to solve all the libc legacy, that is simply not a realistic way forward for C2 at the moment. So there
will be different versions of C2 libc for different platforms. Since C2 does *not* target Windows, most *nix systems don't differ
too much to cause real problems.. 

[kyle]

I ran across this blog post:

https://thefeedbackloop.xyz/thoughts-on-dependency-hell-is-np-complete/

I thought it was particularly interesting since the thread has drifted to package management.  Since C2 does its own management, it might be informative to see what others are doing.