Build system design

Design brainstorming on

How to structure a system to build C, C++ and Java, to support various dependencies and reduce/remove manual configurations to a minimum ?
How to make the build cache mostly invisible ?

The structure and formats is not ready yet, but it will look something like Delibay (course repos + course info + skills list + exos list per skills) and one folder per exo.

cppexos # exos repository, C++ as example here
  course.toml # define your course info
  skills.toml # define your skills info and order
  structs # first skill
    exos.toml # exos definition in this skill
    mega-dog # first exo in this skill
      dog.cpp
    unit-dog # second exo with unit tests in GoogleTest
      dog.cpp
  pointers # second skill
    exos.toml
    on-functions # first exo in this skill with 3 files
      libparse.c
      libparse.h
      main.cpp
    debug # second exo in this skill with one file
      crash.cpp

Build challenges

Problem 1: How to detect project type ?
1. How to avoid the need to define the compilation type ? How to guess it instead ?
Problem 2: How to manage trivial and non trivial build situations ?
1. For trivial situations, how should we name the target ?
2. How to define some dependencies like GoogleTest, Unity or other test runners or libraries
3. How to install those dependencies ? How to accept xmake prompt to install ?
Problem 3: How to make build system almost invisible ?
1. Having 3 additional elements per exo folder is too much noise (xmake.lua, .xmake and build), it would better to have most only build related elements at root of the repository
2. We cannot have a common flat build folder for all exos, because it will create strange errors. We cannot trash it before each exo, because we would loose the big speed improvement of build cache when running all exos or running an exo done in the past.
3. How to differentiate generated and manually written build configurations ?

Build strategies

Solution to problem 1: Using existing configuration or guess how to build trivial cases

Define ExoType = None
If the exo folder contains a xmake.lua, it is used to build: ExoType = xmake
Otherwise, if it contains any .c or .cpp file: ExoType = xmake
Otherwise, if it contains any .java, using javac strategy: ExoType = java

If ExoType == None at this point, throw an error because this is not possible to build this exo...

Solution to problem 2:

If ExoType == java, only trivial situations are supported: with javac directly, running the following command javac -d path/to/build/folder Main.java. The main file should be named Main.java. It will be ran with java -classpath path/to/build/folder/ Main.

If ExoType == xmake and there is no existing xmake.lua, we create it on the fly (only the first comment will be included)

-- Autogenerated by PLX based on guess: xmake + cpp + c. Do not edit directly.
target("exo")
add_files("**.cpp") -- add this line only when .cpp files have been detected
add_files("**.c") -- add this line only when .c files have been detected
-- it's possible to have both C and C++ at the same time but only one `main()` function

In case, we define in exo metadata external libraries or xmake packages (here pcosynchro as library and gtest as package), we dynamically generate xmake instructions add_requires+add_packages and add_links.
```
add_requires("gtest")

target("exo")
add_files("**.cpp")
add_packages("gtest")
add_links("pcosynchro")
```
If this is not possible to solve the build situation with the above possilities, the teacher needs to create xmake.lua by hand. The target must be also be named exo so PLX can detect and run it via xmake run exo.

Solution to problem 3: Group all build folders in a single folder build at root of repos:

Xmake can use a specific config build folder and xmake.lua, javac support a custom output folder, so we make build files almost invisible. This also has the advantage of removing ambiguity on if a xmake.lua has been written by a teacher or has been dynamically generated, as they would be located in different folders: the dynamic config inside the build/... structure, the hand written one in the exo folder.

Global overview

Here is an overview example, considering the previous structure but this time including build files

cppexos
  course.toml
  skills.toml
  structs
    exos.toml
    mega-dog
      dog.cpp
    unit-dog
      dog.cpp
      xmake.lua # special case with need of hand written config
  pointers
    exos.toml
    on-functions
      libparse.c
      libparse.h
      main.cpp
    debug
      crash.cpp

  ...

  .gitignore # must contains "build" folder
  build # common build folder, same structure as above inside,
        # but with build config and cache instead of code
    structs
      mega-dog
        xmake.lua # dynamically generated config file
        ## Generated by xmake for this specific exo
        .xmake
        build
    pointers
      on-functions
        xmake.lua # dynamically generated config file
        ## Generated by xmake for this specific exo
        .xmake
        build
          ...

Xmake example in C++: Let's say we are doing the structs/mega-dog exo editing dog.cpp, here are the steps behind the scenes

Compilation
1. We detect this exo has no existing xmake.lua but has .cpp files so the exo type is xmake.
2. Intermediate folders are created for path build/structs/dog if necessary
3. The trivial config is created under build/structs/dog/xmake.lua
```
-- Autogenerated by PLX based on guess: xmake + cpp. Do not edit directly.
target("exo")
add_files("**.cpp")
```
4. PLX runs the following command xmake build -F build/structs/dog/xmake.lua -P structs/dog/ to indicate source file and build config file.
Execution
1. PLX runs the following command xmake run exo -F build/structs/dog/xmake.lua

Java example: Let's say we are doing the try-catch/except-me exo editing Main.java, here are the steps behind the scenes

cppexos
  course.toml
  skills.toml
  try-catch
    exos.toml
    except-me
      Main.java
      Person.java
      Party.java
  build
    try-catch
      except-me
        Main.class # generated by javac

Compilation
1. We detect this exo contain *.java files so the exo type is java.
2. Intermediate folders are created for path build/try-catch/except-me/ if necessary
3. PLX runs javac -d build/try-catch/except-me/ Main.java
Execution
1. PLX runs java -classpath build/try-catch/except-me/ Main

PLX docs

Build system design

Build challenges

Build strategies

Global overview