Topic: Git repo with CMake available

I added the CMake build system I have been using for the past year to my development repo
https://github.com/Micket/oofem

Its been tested on linux (workstation and computational cluster) and on windows (visual studio). (although I haven't tried out PETSc on windows yet, since PETSc build system does its own thing)

Linux build instructions;

  • mkdir path/to/build/

  • cd path/to/build/

  • ccmake path/to/oofem/

  • make

  • make apidoc

  • ctest

Windows build instructions;

  • Download and install CMake

  • Open CMake and select the oofem directory as the source directory, make a new directory elsewhere for the build

  • Configure once, make choices, configure again, then generate the visual studio scripts

  • Open project file with visual studio

  • Build the "test" target, which compiles then runs the test-suite.

Haven't tried Mac either, but it should be straight forward.

Some advantages

  • Cross platform (e.g. you can actually configure (with full options) on windows, no static visual studio project file)

  • Includes CTest (run test suite conveniently on all platforms)

  • Includes CPack (make source or binary packages (deb, rpm, tgz, zip))

  • Simple and understandable build scripts: https://github.com/Micket/oofem/blob/ma … eLists.txt

  • Less likely to cause merge conflicts in version control when new files are added: https://github.com/Micket/oofem/blob/ma … eLists.txt

  • Far better console output when compiling and testing, errors and warnings stand out

  • Also has option to build the python bindings (although I'm not 100% on how one should construct the libraries)

  • Could easily integrated with CDash, which is nice, for example http://open.cdash.org/index.php?project=Midas

  • "make source_package" for source releases (although it doesn't apply the release filter yet)

2

Re: Git repo with CMake available

I was not aware of cmake capability to build binary and source packages (*.deb and *.rpm). This is a strong argument for cmake.
I think that many end-users will welcome binary packages. If the multiple targets are supported by cmake, then I think we can migrate to cmake.
It would be nice if you can share the work and commit cmake patch into repository. A short wiki page  cmake for dummies (like me) would be helpful to make the transition less paintfull.

Borek

Re: Git repo with CMake available

Multiple targets in CMake would be done by simply configuring in different directories (CMake does out-of-tree builds)
I have 3 of them set up myself
~/build/debug
~/build/release
~/build/parallel
all configured from the source directory in ~/projects/oofem/

I will make a wiki page.

(Disclaimer; I have only tested the *.deb package, and petsc 3.3 is still missing from debian)

Re: Git repo with CMake available

I wrote a small wiki page;
http://www.oofem.org/wiki/doku.php?id=cmake

Re: Git repo with CMake available

Is it OK if I push the CMake scripts to the developer branch?

6

Re: Git repo with CMake available

Yes, I am just thinking about removing configure type of installation at the same time, to avoid any inconsistency. Could you remove the old stuff as well as a part of your commit? The wiki tutorial is great. thank you.

Re: Git repo with CMake available

Ok, so I went through and made sure OOFEG was compiling (had to add some libraries). Not sure how to use it, but it compiles and links fine.

I also haven't really tried out the python bindings yet (but it does compile and link just fine). I need to tell cmake to bake a shared library out of everything.
Preferably, I would want to take everything from sm, fm, tm, oofemlib, and always make a liboofem.[a,so] out of it, then link the executeable to that.
That way, python bindings would just be one more optional feature of the oofem library, and not a separate entity.

(also, to avoid confusion, i renamed the "oofemlib" module to "core" inside the build scripts, and eventually, I was thinking about renmaing the actual src/oofemlib folder as well)

I cleared out the autotools build scripts. If all goes well, I'll push to the develop branch tomorrow

Re: Git repo with CMake available

A small update.

1. I have problems getting the python bindings to work. I can compile and link against boost_python + libpython fine, but it segfaults when I try to run anything linked to boost_python. (I have python bindings as an optional module for "liboofem.so" to which i also link the ordinary oofem executable). And, it actually segfaults during startup (before main()! ).

2. The nicest way to create a liboofem.so (which includes everything but main.C/oofeg.C) and also liboofem.a is to use a quite new feature in CMake (see http://www.cmake.org/pipermail/cmake-de … 03700.html )
This was introduced in CMake 2.8.8 (for comparison. Debian Stable is currently at 2.8.2 and Ubuntu 12.10 is at 2.8.9 )
So I'm a bit torn on this (I noticed how new the feature was after I implemented it).
I'll try to see if i can at least make a fallback for slightly older versions of CMake.

Re: Git repo with CMake available

I believe everything is in working order now, so I thought I'd push to develop branch. I just wanted to give everyone a heads up. I'll push it tomorrow evening unless there are any objections.

There are now options for compiling static or shared libraries (or both), and oofeg+oofem can be compiled with the same configuration, and they link to the same shared library (which includes all modules, so libfm.a libsm.a libtm.a doesn't exists anymore, just a big library for everything).
Internally, i call oofemlib -> core, to not confuse it with the new main target: liboofem.[a|so] which contains everything.

The modules have cross-dependencies and such so it doesn't make much sense in trying to build them in different libraries.



(I'm still not sure exactly how the python bindings are should be compiled).

Re: Git repo with CMake available

Alright, so I just pushed to the develop branch.
If anyone has any trouble with the cmake build scripts, let me know and I'll get on it ASAP.

Re: Git repo with CMake available

I have tested the new cmake setup in the master branch on my mac.
No problem with oofem once I set shared libraries to off.
However, cannot link oofeg. See message below:

Undefined symbols for architecture x86_64:
  "_XtNameToWidget", referenced from:
      set_layer_on_off(EView*, char*, CkitPoint3Struct*) in oofeg.C.o
      updateDefPlotFlag()     in oofeg.C.o
      straincompPlot(_WidgetRec*, void*, void*)in oofeg.C.o
      stresscompPlot(_WidgetRec*, void*, void*)in oofeg.C.o
      apply_layer_update(_WidgetRec*, void*, void*)in oofeg.C.o
      apply_mat_reg_filter(_WidgetRec*, void*, void*)in oofeg.C.o
      _apply_change in libesi.a(simple1.o)
      ...

And many more. At the end:

ld: symbol(s) not found for architecture x86_64

collect2: ld returned 1 exit status
make[2]: *** [oofeg] Error 1
make[1]: *** [CMakeFiles/oofeg.dir/all] Error 2
make: *** [all] Error 2

Any idea what one could try.

Re: Git repo with CMake available

Hi gustel. What type of problem do you run into when compiling with shared libraries?

I don't have a mac available to work on, but I would guess that its some part of X11 missing from linking.
If you run
"make VERBOSE=1 oofeg"
you can see the command that it tries for linking.
For me on linux is looks like

/usr/bin/c++    -Wall -fPIC -g -DDEBUG    CMakeFiles/oofeg.dir/src/main/oofeg.C.o  -o oofeg -rdynamic liboofem.a liboofem.a /home/micket/projects/krysl/Elixir/src/libesi.a /home/micket/projects/krysl/Elixir/src/libelixir.a /home/micket/projects/krysl/Ckit/src/libckit.a -lSM -lICE -lX11 -lXext -lXaw

Its a bit difficult to check, but I thought that "_XtNameToWidget" as found in libXaw.so, but appearantly not on mac. Perhaps its finding a 32bit version of the library instead of 64bit?

Have you  been able to compile OOFEG before?
If so, could you possibly check the flags used for linking that version?

Re: Git repo with CMake available

For oofem I get this error message when i just run the default settings:

===============================================================
===============================================================
Scanning dependencies of target liboofem
[100%] Building CXX object CMakeFiles/liboofem.dir/src/main/usrdefsub.C.o
[100%] Building CXX object CMakeFiles/liboofem.dir/src/main/classfactory.C.o
Linking CXX shared library liboofem.dylib
Undefined symbols for architecture x86_64:
  "oofem::classFactory", referenced from:
      oofem::CreateUsrDefElementOfType(char const*, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefRandomFieldGenerator(oofem::classType, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefRandomFieldGenerator(char const*, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefNonlocalBarrierOfType(oofem::classType, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefNonlocalBarrierOfType(char const*, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefLoadTimeFunctionOfType(oofem::classType, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefLoadTimeFunctionOfType(char const*, int, oofem::Domain*)in usrdefsub.C.o
      ...
  "oofem::oofem_errLogger", referenced from:
      oofem::CreateUsrDefNodalRecoveryModel(oofem::NodalRecoveryModel::NodalRecoveryModelType, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefGeometry(oofem::classType)   in usrdefsub.C.o
      oofem::CreateUsrDefEnrichmentFunction(oofem::classType, int, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefEnrichmentItem(oofem::classType, int, oofem::XfemManager*, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefPatch(oofem::Patch::PatchType, oofem::Element*)in usrdefsub.C.o
      oofem::CreateUsrDefMesherInterface(oofem::MeshPackageType, oofem::Domain*)in usrdefsub.C.o
      oofem::CreateUsrDefMaterialMappingAlgorithm(oofem::MaterialMappingAlgorithmType)    in usrdefsub.C.o
      ...
  "oofem::oofem_logger", referenced from:
      oofem::DynCompRow::ILUPYourself(int, double)in dyncomprow.C.o
      oofem::DynCompRow::buildInternalStructure(oofem::EngngModel*, int, oofem::EquationID, oofem::UnknownNumberingScheme const&)in dyncomprow.C.o
      oofem::SparseMtrx::printYourself() const in dyncomprow.C.o
      oofem::SparseMtrx::printStatistics() const in dyncomprow.C.o
      oofem::SparseMtrx::writeToFile(char const*) constin dyncomprow.C.o
      oofem::DynCompCol::buildInternalStructure(oofem::EngngModel*, int, oofem::EquationID, oofem::UnknownNumberingScheme const&)in dyncompcol.C.o
      oofem::DynCompCol::printStatistics() const in dyncompcol.C.o
      ...
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
make[3]: *** [liboofem.dylib] Error 1
make[2]: *** [CMakeFiles/liboofem.dir/all] Error 2
make[1]: *** [CMakeFiles/oofem.dir/rule] Error 2
make: *** [oofem] Error 2
=======================================================
=======================================================

By switching off shared libraries it works.

I will look now at the flags for oofeg. It worked before the transition to cmake.
So, should be possible to sort it out.

Re: Git repo with CMake available

For oofeg the VERBOSE gives me this for cmake and the old way:


/opt/local/bin/cmake -E cmake_link_script CMakeFiles/oofeg.dir/link.txt --verbose=1
/usr/bin/c++    -Wall -fPIC -g -DDEBUG -Wl,-search_paths_first -Wl,-headerpad_max_install_names   CMakeFiles/oofeg.dir/src/main/oofeg.C.o  -o oofeg  liboofem.a liboofem.a /Users/gustel/Elixir/src/libesi.a /Users/gustel/Elixir/src/libelixir.a /Users/gustel/Ckit/src/libckit.a /opt/local/lib/libSM.dylib /opt/local/lib/libICE.dylib /opt/local/lib/libX11.dylib /opt/local/lib/libXext.dylib /opt/local/lib/libXaw.dylib

The old way that worked is this:

g++ -g -D__OOFEG -D__OOFEMLIB_MODULE -D__SM_MODULE -D__TM_MODULE -D__CEMHYD_MODULE -D__FM_MODULE -D__TINYXML_MODULE -DTIXML_USE_STL   -o /Users/gustel/oofem.git/targets/oofeg-debug/bin/oofeg main.o oofeg.o usrdefsub.o classfactory.o /Users/gustel/oofem.git/targets/oofeg-debug/lib/fm/libfm.a /Users/gustel/oofem.git/targets/oofeg-debug/lib/tm/libtm.a /Users/gustel/oofem.git/targets/oofeg-debug/lib/sm/libsm.a /Users/gustel/oofem.git/targets/oofeg-debug/lib/oofemlib/liboofemlib.a /Users/gustel/Elixir/src/libesi.a /Users/gustel/Elixir/src/libelixir.a /Users/gustel/Ckit/src/libckit.a -lm   -lSM -lICE  -L/usr/X11/lib -R/usr/X11/lib -lXaw -lXmu -lXt -lXext -lX11  -lm

Re: Git repo with CMake available

So it seems you have some X11 libraries in /usr/X11/lib that CMake finds by default.
Could you verify this by checking for example

file /usr/X11/lib/libXaw.dylib

But you probably want to use the custom libraries in /opt/local/lib/ instead, so I suppose I should add a parameter for that in the cmake scripts.
I added a parameter X11_DIR which you can set when configuring (/opt/local/lib/ should work in your case).
Could you try out the updated CMakeLists.txt attached here to see if that sorts out the problems?


For the dynamic library problem, could you again check the actual link flags using VERBOSE?

Post's attachments

CMakeLists.txt 23.92 kb, 6 downloads since 2012-11-30 

You don't have the permssions to download the attachments of this post.

Re: Git repo with CMake available

It is actually the other way around. I should use /usr/X11/lib/ as this worked in the old approach.
However, when I add this path for the X11 dir using the new CMakeLists.txt I get this error.

=======================================================
=======================================================
CMake Error at CMakeLists.txt:161 (find_package):
   Could not find a package configuration file provided by "X11" with any of
   the following names:

     X11Config.cmake
     x11-config.cmake

   Add the installation prefix of "X11" to CMAKE_PREFIX_PATH or set "X11_DIR"
   to a directory containing one of the above files.  If "X11" provides a
   separate development package or SDK, be sure it has been installed.
========================================================
=========================================================

Re: Git repo with CMake available

Hmm. Ok.
Long explanation;
So I use the CMake module
find_package(X11 ...)
to find the suitable X11-libraries to link with. I was incorrect when i tried to add to the PATH option did for find_package (instead of adding a search path for the libraries, it adds a search path for the cmake file)
find_package will in the case on my computer run /usr/share/cmake-2.8/FindX11.cmake
which uses some hardcoded values;

  SET(X11_LIB_SEARCH_PATH
    /usr/pkg/xorg/lib
    /usr/X11R6/lib
    /usr/X11R7/lib
    /usr/openwin/lib
  )

and then runs the following find_library commands for all the X11 libs

  FIND_LIBRARY(X11_ICE_LIB ICE               ${X11_LIB_SEARCH_PATH})
  FIND_LIBRARY(X11_SM_LIB SM                 ${X11_LIB_SEARCH_PATH})
....

The find_library function also includes the system default paths (since none of the specified paths match), so I can only imagine that you actually list /opt/local/lib/ somewhere that the compiler can find automatically.
Luckily it should be possible to tell CMake what to use for these default paths and bypass the /opt/local/lib/ path

You need to set

CMAKE_LIBRARY_PATH=/usr/lib/X11/

which should then hopefully come first
You can set it by doing

ccmake -DCMAKE_LIBRARY_PATH=/usr/lib/X11/ /path/tio/oofem/

Or if you use the CMake GUI then you can do "Add entry"

Note:

  • You might need to remove the cache to make it forget about the old libs in /opt/

  • If it insist on finding /opt/ libs first, then you could force it to ignore that path by using the CMAKE_IGNORE_PATH variable

  • You also might need to add set CMAKE_INCLUDE_PATH as well

Short version;
I made a new CMakeLists.txt that you can try out.

Post's attachments

CMakeLists.txt 23.95 kb, 2 downloads since 2012-11-30 

You don't have the permssions to download the attachments of this post.

Re: Git repo with CMake available

Thanks for the additional info.
I manged to use my usr/X11/lib. Bu this did not solve my problem.
Then I changed to g++ thinking that this might make a difference. However, I still get the same errors.

My linking info is:

/opt/local/bin/cmake -E cmake_link_script CMakeFiles/oofeg.dir/link.txt --verbose=1
/usr/bin/g++   -g -DDEBUG -Wl,-search_paths_first -Wl,-headerpad_max_install_names   CMakeFiles/oofeg.dir/src/main/oofeg.C.o  -o oofeg  liboofem.a liboofem.a /Users/gustel/Elixir/src/libesi.a /Users/gustell/Elixir/src/libelixir.a /Users/gustel/Ckit/src/libckit.a /usr/X11/lib/libSM.dylib /usr/X11/lib/libICE.dylib /usr/X11/lib/libX11.dylib /usr/X11/lib/libXext.dylib /usr/X11/lib/libXaw.dylib

Re: Git repo with CMake available

Hmm. The only difference I can spot right now is the following libraries (which you had linked in the old version, and not with cmake);

-lm -lXmu -lXt

which weren't needed on linux. Did you still get the same error message with the new libs specified?

Could you just manually try out the following command
/usr/bin/g++   -g -DDEBUG -Wl,-search_paths_first -Wl,-headerpad_max_install_names   CMakeFiles/oofeg.dir/src/main/oofeg.C.o  -o oofeg  liboofem.a liboofem.a /Users/gustel/Elixir/src/libesi.a /Users/gustell/Elixir/src/libelixir.a /Users/gustel/Ckit/src/libckit.a /usr/X11/lib/libSM.dylib /usr/X11/lib/libICE.dylib /usr/X11/lib/libX11.dylib /usr/X11/lib/libXext.dylib /usr/X11/lib/libXaw.dylib /usr/X11/lib/libXmu.dylib  /usr/X11/lib/libXt.dylib -lm

And if that doesn't work, can you please paste the output from

file /lib/libXaw.dylib

and also the linker errors that you get

Re: Git repo with CMake available

Adding the three libraries worked.
Is there any way to add this to the CMakeList.txt so that the compiling oofeg is more automatic?

Re: Git repo with CMake available

Yes, its easily done, I just didn't know I which ones I had to add by.

I have another new CMakeLists.txt for you to try out. I didn't explicitly add "-lm" since it should be included by default by the compiler (even without X11, the math lib is needed)

Post's attachments

CMakeLists.txt 24.05 kb, 8 downloads since 2012-12-01 

You don't have the permssions to download the attachments of this post.

Re: Git repo with CMake available

gustal, please let me know if the new cmakelists.txt works automatically, and I will commit.

Re: Git repo with CMake available

Sorry, I thought I had replied to it already.

I still need to switch shared libs to off and switch static libs on.
Also, when I configure with ccmake oofeg I get an output which reads

/opt/local/lib/libXmu.dylib/opt/local/lib/libXt.dylib

However, when I continue to generate, I am able to make oofeg.


Also, during the compilation, I get these warnings

In file included from /Users/gustel/Elixir/include/Econfig.h:43,
                 from /Users/gustel/Elixir/include/Etypes.h:42,
                 from /Users/gustel/Elixir/include/Esimple.h:70,
                 from /Users/gustel/oofem.git/src/oofemlib/oofeggraphiccontext.h:63,
                 from /Users/gustel/oofem.git/src/oofemlib/femcmpnn.h:58,
                 from /Users/gustel/oofem.git/src/oofemlib/element.h:38,
                 from /Users/gustel/oofem.git/src/fm/fmelement.h:38,
                 from /Users/gustel/oofem.git/src/fm/cbselement.h:38,
                 from /Users/gustel/oofem.git/src/fm/tr1_2d_cbs.h:38,
                 from /Users/gustel/oofem.git/src/fm/tr1_2d_cbs.C:35:
/Users/gustel/Elixir/include/Ecfg.h:133:1: warning: this is the location of the previous definition

Now that I know what to do, it is fine for me. However, maybe for other mac users the shared library issue might not be obvious?

Re: Git repo with CMake available

I accidentally left a debug message in the cmake file, to make sure it found the requested libs.

The shared library is probably a separate issue (I suspect you'd run into similar problems if you tried linking with libpetsc or some other static lib as well. I'll try to sort it out. Either way, I think I'll change the default value for shared lib to OFF.

The warnings, well, should be warnings. Elixir includes its own config header with things like HAVE_UNISTD_H, which we need to configure in OOFEM as well.
They should be harmless, but of course, they are distracting from real problems, so hopefully it can be sorted out.
Preferably, Elixir shouldn't push such a config header into other projects (problem is from Econfig.h which includes Ecfg.h).
Where these warnings not there in the old version, when compiled with -Wall? I can't imagine that this is directly caused by the CMake script.

Re: Git repo with CMake available

I don't know where the warning comes form. I don't get it if I use -Wall in the old version.
But as you say, it is only a harmless warning.
Can you add the other change with the shared and static lib to the repo?