Topic: More on modularity (classfactory)

Currently, to add a new component in OOFEM, you need to add it to CMakeLists.txt, class factory and classtype.
I've discussed classType in the other thread, and CMakeLists I think is acceptable.

I've experimented a little with distributing the components into their own files, and I have introduced all the functions and macros necessary to do this. All that's needed for each component is a single line like:
REGISTER_Element(MyElement);
in its source file.
With this, you could even link a separate shared library, which would register itself with the classfactory, without modifying any code inside OOFEM itself.

It works fine when using shared libraries, but with static libraries, internally used symbols are optimized away. There are compiler specific flags which forces all symbols to be included when linking static libraries, but I don't want to rely on those (afaik there is no way to do it with MSVC).
Linking all the objects files into a single executeable also works.

I'm planning on changing the CMakelists to allow for these two methods, getting rid of the static library completely.
Since also the python interface requires a shared library, I think we should consider fixing this for windows as well. The problem here is that windows requires all external functions to be declared manually. This means we need to do something along the lines of

// Compiler flag: -D__OOFEM_API __declspec(dllexport)
class __OOFEM_API MyClass

in at least all the methods that main.C uses (which fortunately isn't that many)
I'm not sure if this is the case for mingw as well.

Re: More on modularity (classfactory)

A followup on the linking business. Things would be a lot simpler if we simply enforced CMake version 2.8.8, which had a new feature which makes dynamic libraries and such much simpler.

Currently, I've added some backwards compatibility mode for 2.8.7 (which is the version in ubuntu LTS), but for windows you need a newer version (or we need to fix the dynamic linking on windows, which is a hassle).
Also, the python bindings won't work unless you have CMake >= 2.8.8.

But with this, one can now register elements outside of classfactory itself, at link time.  I've converted the smaller ***classfactory.h-files to use the distributed approach.

Re: More on modularity (classfactory)

The new changes to the CMakeLists mean that oofeg on Mac OSX does not work anymore.
We had similar issue when cmake was introduced:  http://www.oofem.org/forum/viewtopic.php?pid=3156#p3156
I have attached output for shared libraries switched off, which I used previously.
It also does not work if shared libraries are on.

Post's attachments

verboseOff.txt 52.17 kb, 2 downloads since 2013-05-03 

You don't have the permssions to download the attachments of this post.

Re: More on modularity (classfactory)

Thank you for reporting the issue. I had a typo when linking oofeg (wrote "oofem" instead of "oofeg")
Hopefully, this should be fixed now. Let me know how it turns out.

I'm also interested in getting shared libraries to work on OSX (and on windows), but I haven't had access to either machines to it is hard to fix. Patches would be appreciated, because shared libraries would be very important (for the python bindings etc).

Edit: I'll try to compile OOFEM on a mac on monday (borrowing it from a colleague).

Re: More on modularity (classfactory)

"Shared libraries off" works now fine. Thanks.

Re: More on modularity (classfactory)

I have distributed all the components into respective classes now. This lets us have a truly modular compilation. Instead of #ifdef-ing components, one can simply just turn on/off compilation and they won't register themselves in the classfactory. Several inline compiler definitions have been made redundant (which is a good sign that the code is modular).


There are still some components which use static enums (error estimator, sparse matrix, linear solvers). This is inflexible if some third party wants to add a new component. It is also inconvenient to use in input files, as you have to specify a number instead of saying "lstype petsc" in input files. It is simple enough to change this, but it would make old input files incompatible, so I've left it for now.

If we ever consider renaming elements consistently, we should take this into account then (a long with other changes, like dropping the domain-type record). None of these changes should be a problem for a straight-forward conversion script.

7

Re: More on modularity (classfactory)

I have experienced following error: when configuring oofem with both sm and tm modules, I always get following:

src/sm/libsm.so: undefined reference to `oofem::HeMoTKMaterial::inverse_sorption_isotherm(double)

This seems to be related to the dynamic linking. Any idea how to solve this?

BTW: seems also that latticedamage2d.C model does not compiles witthout tm module. I will contact its responsible developper.

Re: More on modularity (classfactory)

I never encountered this problem (and I have always compiled with all modules).
I suspect that you are using CMake version 2.8.7 (which still should work). These changes might require you to remove the CMakeCache, so that might be it. I'll see if I can find the root of the linking problem.

For the second issue, I think latticetransportelement.h belongs in the core module which should take care of the compilation error.

9

Re: More on modularity (classfactory)

I can confirm that this is related to cmake version. The 2.8.7 version will fail, producing the error described above, while 2.8.8 works fine.

Re: More on modularity (classfactory)

I can't reproduce the linking problem. If clearing the CMakeCache isn't working, then I would need to see the verbose output (make VERBOSE=1) from the failed linking command.

11

Re: More on modularity (classfactory)

The output from the linking phase is attached.

Post's attachments

ee 5.46 kb, 3 downloads since 2013-05-06 

You don't have the permssions to download the attachments of this post.

Re: More on modularity (classfactory)

So the main executeable fails. The actual linking command looks identical to what I get (which works here).
The missing symbol should be inside src/tm/libtm.so, and it is for me

$ nm src/tm/libtm.so  | grep inverse_sorption_isotherm
425:00000000000641d0 T _ZN5oofem14HeMoTKMaterial25inverse_sorption_isothermEd

If its there, then I'm not sure why the dynamic linking wouldn't accept it.

If not, then perhaps the output from

rm src/tm/libtm.so
make VERBOSE=1

could reveal something. But hemoktmat isn't conditionally compiled, so I can't see why it wouldn't be in there.

13

Re: More on modularity (classfactory)

The libtm.so contains the inverse_sorption_isotherm symbol:

$ nm src/tm/libtm.so | grep inverse_sorption_isotherm
000000000006021e T _ZN5oofem14HeMoTKMaterial25inverse_sorption_isothermEd

I don't think the problem is in libtm, as the error message from the linker says

src/sm/libsm.so: undefined reference to `oofem::HeMoTKMaterial::inverse_sorption_isotherm(double)'
collect2: ld returned 1 exit status

this seems like the symbol is expected in libsm.so, the symbol lookup gives following:

$ nm src/sm/libsm.so | grep inverse_sorption_isotherm
0000000000276774 T _ZN5oofem10B3Material25inverse_sorption_isothermEd
00000000002f26ca T _ZN5oofem11MPSMaterial25inverse_sorption_isothermEd
                 U _ZN5oofem14HeMoTKMaterial25inverse_sorption_isothermEd
000000000027ae02 T _ZN5oofem15B3SolidMaterial25inverse_sorption_isothermEd

there is one unresolved symbol related to inverse_sorption_isotherm, which could be resolved by linking with libtm.so, perhaps the order of linking is important?

Re: More on modularity (classfactory)

The symbol shouldn't need be defined, and shouldn't need to be there either. For static libraries, cyclic depedencies is a bit of an issue (usually requiring you to link the libraries multiple times of use special linking options, but for shared libraries this should never be necessary.

As far as I can tell, I have the exact same symbols, and my linking order is also the same, and it works:

// My linking command (works here)
/usr/bin/c++    -Wall -fPIC -g -DDEBUG    CMakeFiles/oofem.dir/src/main/main.C.o  -o oofem -rdynamic src/oofemlib/libcore.so src/fm/libfm.so src/tm/libtm.so src/sm/libsm.so -ldl -Wl,-rpath,/home/micket/projects/build/test/src/oofemlib:/home/micket/projects/build/test/src/fm:/home/micket/projects/build/test/src/tm:/home/micket/projects/build/test/src/sm:
// From your uploaded file:
/usr/bin/c++    -Wall -fPIC -g -DDEBUG    CMakeFiles/oofem.dir/src/main/main.C.o  -o oofem -rdynamic src/oofemlib/libcore.so src/fm/libfm.so src/tm/libtm.so src/sm/libsm.so -ldl -Wl,-rpath,/home/bp/oofem/build/oofem-debug/src/oofemlib:/home/bp/oofem/build/oofem-debug/src/fm:/home/bp/oofem/build/oofem-debug/src/tm:/home/bp/oofem/build/oofem-debug/src/sm:

and my nm dump looks identical as well

$ nm src/tm/libtm.so | grep inverse_sorption_isotherm
425:0000000000061530 T _ZN5oofem14HeMoTKMaterial25inverse_sorption_isothermEd
$ nm src/sm/libsm.so | grep inverse_sorption_isotherm
295:000000000028238c T _ZN5oofem10B3Material25inverse_sorption_isothermEd
1213:0000000000307304 T _ZN5oofem11MPSMaterial25inverse_sorption_isothermEd
2140:                 U _ZN5oofem14HeMoTKMaterial25inverse_sorption_isothermEd
2372:0000000000287134 T _ZN5oofem15B3SolidMaterial25inverse_sorption_isothermEd

I'm running short on ideas here. Everything looks alright to me, and the identical setup runs fine here. A shared library using a symbol from another shared library shouldn't be a problem if all the libraries are used when linking.

I was using all default options. The linking command for each module looks like this

/usr/bin/c++  -fPIC  -Wall -fPIC -g -DDEBUG   -shared -Wl,-soname,libsm.so -o libsm.so [list of many many object files here]

I tried to create a minimal example which replicates this scenario:

// In fileA.c
int foo() {
    return 1;
}
// In fileB.c
int foo();
int bar() {
    return foo() + 1;
}
// In main.C
int bar();
int main() {
    return bar() + 2;
}

and I compile this little sequence as

c++ -fPIC -shared -Wl,-soname,fileA.so -o libfileA.so fileA.C
c++ -fPIC -shared -Wl,-soname,fileB.so -o libfileB.so fileB.C
c++ -g main.C -o test libfileA.so libfileB.so

The reverse order or the shard libraries also works.
Symbol dumping also shows the same scenario with the function foo();

$ nm libfileA.so | grep foo
6:0000000000000670 T _Z3foov
$ nm libfileB.so | grep foo
7:                 U _Z3foov

So I'm still unable to replicate any errors.

Re: More on modularity (classfactory)

Borek, did you manage to sort out the compilation problems?


I also wanted to mention that I've gone through (hopefully) all components and implemented giveInputRecordName. It now uses the same defined string as the classfactory uses for registration (i.e. no duplicated information). It rarely matched with giveClassName, so I removed the default implementation of FEMComponent::giveInputRecordName, forcing elements to overload it.