I’ve used Make for a lot for small projects, but for larger ones, it was just too tedious. Until recently, there were four things I wanted my build system to do for me that I hadn’t figured out how to do in Make:
- Out-of-source builds (object files get dumped in a separate directory from the source)
- Automatic (and accurate!) header dependencies
- Automatic determination of list of object/source files
- Automatic generation of include directory flags
Here is a simple Makefile that will do all these things and works with C, C++, and assembly:
TARGET_EXEC ?= a.out
BUILD_DIR ?= ./build
SRC_DIRS ?= ./src
SRCS := $(shell find $(SRC_DIRS) -name *.cpp -or -name *.c -or -name *.s)
OBJS := $(SRCS:%=$(BUILD_DIR)/%.o)
DEPS := $(OBJS:.o=.d)
INC_DIRS := $(shell find $(SRC_DIRS) -type d)
INC_FLAGS := $(addprefix -I,$(INC_DIRS))
CPPFLAGS ?= $(INC_FLAGS) -MMD -MP
$(BUILD_DIR)/$(TARGET_EXEC): $(OBJS)
$(CC) $(OBJS) -o $@ $(LDFLAGS)
# assembly
$(BUILD_DIR)/%.s.o: %.s
$(MKDIR_P) $(dir $@)
$(AS) $(ASFLAGS) -c $< -o $@
# c source
$(BUILD_DIR)/%.c.o: %.c
$(MKDIR_P) $(dir $@)
$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@
# c++ source
$(BUILD_DIR)/%.cpp.o: %.cpp
$(MKDIR_P) $(dir $@)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $< -o $@
.PHONY: clean
clean:
$(RM) -r $(BUILD_DIR)
-include $(DEPS)
MKDIR_P ?= mkdir -p
Not too bad!
Also, if you don’t care about out-of-source builds, you can use this even simpler Makefile, which takes advantage of the built-in implicit rules:
TARGET ?= a.out
SRC_DIRS ?= ./src
SRCS := $(shell find $(SRC_DIRS) -name *.cpp -or -name *.c -or -name *.s)
OBJS := $(addsuffix .o,$(basename $(SRCS)))
DEPS := $(OBJS:.o=.d)
INC_DIRS := $(shell find $(SRC_DIRS) -type d)
INC_FLAGS := $(addprefix -I,$(INC_DIRS))
CPPFLAGS ?= $(INC_FLAGS) -MMD -MP
$(TARGET): $(OBJS)
$(CC) $(LDFLAGS) $(OBJS) -o $@ $(LOADLIBES) $(LDLIBS)
.PHONY: clean
clean:
$(RM) $(TARGET) $(OBJS) $(DEPS)
-include $(DEPS)
To use one of them, put the Make code in a file call Makefile
(make sure the TAB characters get copied! Make is very picky about those) and all of your source and headers in the directory or a subdirectory of ./src
(you can change this directory by changing SRC_DIRS). Then make sure you have CC and CFLAGS set to what you need for your project or just use the Make defaults.
Then type make
.
If you run into issues, running make -d
can be helpful.
Here’s an overview of how it works:
Out-of-Source Builds
I want all the artifacts from a build to end up in some directory (I usually name it “./build”) that’s separate from the source. This makes is easy to do a clean (just rm -rf ./build
) even if other artifacts besides the ones generated via Make end up there. It also makes a lot of other things, such as grep’ing the source, a lot nicer.
To do this in Make, you mostly just need to prepend your output directory to the beginning of your pattern rules. For example, instead of a pattern like: %.o: %.c
, which would map your .c files for .o files in the same directory, you can use $(BUILD_DIR)%.o: %.c
.
Automatic Header Dependencies
Handling the header dependencies is perhaps the most tedious thing about using the classic Make technique. Especially since, if you mess it up, you don’t get any explicit errors–things just don’t get re-compiled when they ought to be. This can lead to .o files having different ideas about what types or prototypes look like.
There is documentation for this here. However, the docs seem to assume that the dependency files are generated in a separate step from the compile step, which complicates things.
If you generate the dependency files as part of the compilation step, things get much simpler. To generate the dependency files, all you have to do is add some flags to the compile command (supported by both Clang and GCC):
-MMD -MP
which will generate a .d file next to the .o file. Then to use the .d files, you just need to find them all:DEPS := $(OBJS:.o=.d)
and then -include them:-include $(DEPS)
Automatic Determination of List of Object/Source Files
First, find all of the source files in the given source directories. The simplest and fastest way I found to do this was to just shell out and use find
.
SRCS := $(shell find $(SRC_DIRS) -name *.cpp -or -name *.c -or -name *.s)
But because Make works backward from the object files to the source, we need to compute all the object files we want from our source files. I basically just prepend a$(BUILD_DIR)/
and append a.o
to every source file path:OBJS := $(SRCS:%=$(BUILD_DIR)/%.o)
And then you can make your target depend on the objects files:-
$(BUILD_DIR)/$(TARGET_EXEC): $(OBJS) $(CC) $(OBJS) -o $@ $(LDFLAGS)
Automatic Generation of Include Directory Flags
I used a similar technique to generate include directory flags. Find all the directories under the given source directories:
INC_DIRS := $(shell find $(SRC_DIRS) -type d)
And then prefix them with a-I
:INC_FLAGS := $(addprefix -I,$(INC_DIRS))
Hope those techniques are helpful for you.
Just use cmake or scons. In 2016, you should not hand write a single makefile !
A simple cmake/sconcs build file will be as simple as the one shown (or even simpler !) and it will definitively will be easier to integrate external dependencies if you have to.
actually, there is more work to be done for cmake: you are writing the cmake config instead of the make file directly, the process is just as lengthy if not worse and requires learning the syntax and steps: https://cmake.org/cmake-tutorial/
I think the “simplicity” in tools like cmake and scons are illusionary. They _are_ often _easier_ to get a project up running quickly, but have _a lot_ more hidden internal complexity. Simple != Easy: https://www.infoq.com/presentations/Simple-Made-Easy
One example where I got bit by scons specifically:
I had a case where I needed to have the compiler run from a different directory than where scons was running it. It’s actually _really hard_ to control the directory where scons runs the compiler from: http://scons.org/doc/2.1.0/HTML/scons-user/x3398.html
Normally this doesn’t matter, but it caused me problems when I was trying to collect code coverage metrics for a project. Gcc/Clang put the paths to the corresponding sources into the generated coverage files and other tools like gcov/lcov use those paths when generating coverage reports.
Our project was quite large and actually used several scons ‘variant_dirs’ for different components. And when the different components were linked together and run, the resulting coverage data had source paths that were all relative to different directories! And there was no way to tell scons to always run gcc from a consistent directory, or to tell gcov/lcov how to untangle the mess.
Nice, I like it. There’s one addition I would like to mention though.
In order to get reproducible linking order (and hence the same binary for multiple runs) I would wrap the ‘find’ call in ‘$(sort …)’ with LC_ALL defined to a specific locale (to avoid locale specific sorting variation). Please also see https://reproducible-builds.org/docs/stable-inputs/ — this will make your downstream packagers’ life a bit easier :)
Note that the tabs in your makefile have been converted to spaces by your blog posting software, which is going to be very confusing to any newbies who try out your makefile as it will fail with a syntax error when they try and invoke make.
yeah :(
Unfortunately I don’t think I can fix that, but I’ll try :(
ok, I think I fixed the tabs, but people still have to watch out for editors replacing the TAB with spaces on paste.
This is actually a really nice makefile :D
The only things I could add would be an example of where in include library link flags.
But this is pretty much my makefile, only using find over wildcard.
Oh, and ctags generation stuff.
Thanks! :D
Using “find” to discover source files means that deleting a source file does not trigger a rebuild. I need to know to manually run “make clean” to avoid the object file fro, the deleted source file being part of my executable.
good point! That’s rather nasty, do you have ideas for how I could fix that?
I don’t think there’s a perfect answer. I’ve seen real-world projects that use the “find” approach and they generally don’t try to handle this problem since it rarely bites you in practice and when it does you can “make clean”.
It might be feasible to try to delete all .o files for which there is no corresponding source file, although that wouldn’t fix the even rarer case of deleting just a header file.
For this, I use something like:
SOURCE = $(shell find src/ -type f | sort | xargs shasum)
.run-always:
.source: .run-always
@echo $(SOURCE) | cmp -s – $@ || echo $(SOURCE) > $@
$(TARGET): .source
@$(CC) …
For this, I use something like:
SOURCE = $(shell find src/ -type f | sort | xargs shasum)
.run-always:
.source: .run-always
@echo $(SOURCE) | cmp -s – $@ || echo $(SOURCE) > $@
$(TARGET): .source
@$(CC) …
How about using built-in function named wildcard? Here’s the link: https://www.gnu.org/software/make/manual/make.html#Wildcard-Function
The main advantage of using find here is that it searches recursively. (wildcard does not)
How could I convert this for Objective-C?
This is a neat MakeFile, and as much as MakeFiles ‘do the job well’ in *nix, I don’t find them to be a scalable solution for multi-platform projects.
As an alternative to CMake, there is also BuildAMation (http://buildamation.com/). It targets desktop platforms, can run a multithreaded command line build, or generate VisualStudio projects, Xcode projects, or even MakeFiles, all from a single declarative description of what output to build, and how those outputs dependent on each other.
It uses C#, so requires Mono on *nix, but it’s pretty fast, and the scripts are in a real programming language that you can debug and profile.
It’s extensible too, so beyond the core, there are packages for common compilers, Qt, Python interpreter, graphics APIs, zeromq, Flex and Bison, to name a few.
Could you publish these Makefile examples on Github with an explicit licence ? If you don’t do that, by the general rules of copyright, it means “all rights reserved” and we can’t use the code.
You are free to use and distribute these Makefile examples however you wish.
If you need an explicit license: https://gist.github.com/jvranish/ae92266fa3039a5e5e8dc4cbc265f225
Great makefile template! Thanks for sharing.
I had to use $(CXX) for the linker rule otherwise I get lots of undefined symbols for C++ methods.
Thanks for the Makefiles really usefull for me.
Anyway when i typed make i received this error:
find: paths must precede expression: …….cpp
and to solve in my case i need to add single quote to…
SRCS := $(shell find $(SRC_DIRS) -name ‘*.cpp’ -or -name ‘*.c’ -or -name ‘*.s’)