Genreflex

This page is work in progress. It is set up to collect the documentation about genreflex in Root6. It is not meant by any means to be official and will be migrated to official root documentation once the work is finished.

Genreflex in root5

In root5, the dictionaries of the classes are generated with genreflex, a python script which takes advantage of gccxml. Basically its input and output are:

• input: a bunch of headers
• output: a c++ source file of the dictionary, optionally a rootmap file.

This dictionary can be then compiled and used for example within the cint interpreter to access method of classes or for IO. This is how one uses genreflex to generate dictionaries of a particular class:

myClass header

class myClass {

public:
   myClass(int i, int j);
   myClass();
   ~myClass();
   int getI();
   void setI(int i);
   int getJ();
   void setJ(int i);
private:
   int m_i;
   int m_j;
};

myClass implementation

#include "myclass.h"

myClass::myClass(int i, int j):
  m_i(i),
  m_j(j){};
  
myClass::myClass():
   m_i(0),
   m_j(0){};

myClass::~myClass(){};

int myClass::getI(){return m_i;};

void myClass::setI(int i){m_i=i;};

int myClass::getJ(){return m_j;};

void myClass::setJ(int j){m_j=j;};

commands to create dictionary and shared library

genreflex myClass.h -o myClass_dict.cpp
g++ myClass.cpp myClass_dict.cpp  `root-config --cflags --libs` -lCintex -lReflex  -shared -o libmyClass.so -fPIC

Note the link to the Cintex and Reflex libraries. The class is very simple and does not need any additional include file. If this would have been the case, genreflex also accepts -I options (which are passed to gccxml. Same holds for -D and -U for example.)

Access methods within cint

root [0] gSystem->Load("libCintex.so");Cintex::Enable();gSystem->Load("libmyClass.so")
(int)0
root [1] myClass a(1,2);a.getI()
(int)1

Genreflex in root6

Reflex is not part of root6: the new Cling infrastructure is able to offer the same functionalities. An executable called genreflex is nevertheless offered with the same api and functionalities as the one of root5 in order to allow a smoother transition to the new root. Basically its input and output are:

• input: a bunch of headers
• output: a c++ source file of the dictionary, its header and a precompiled module (pcm), optionally a rootmap file.

An additional switch has been added, -m, to declare pcms which are to be "subtracted" from the produced pcm. This is done to circumvent some issues linked to the merging of pcms. Let's consider the previous class.

commands to create dictionary and shared library

genreflex myClass.h -o myClass_dict.cpp
g++ myClass.cpp myClass_dict.cpp  `root-config --cflags --libs`  -shared -o libmyClass.so -fPIC

Access methods within cling

root [0] gSystem->Load("libmyClass.so")
(int) 0
root [1] myClass a(1,2);a.getI()
(int) 1

As one can notice, reflex and cintex are not present anymore (by construction!).

The first big project adopting root6 and using genreflex: Gaudi

The nightly builds are put in place by LCG. They can be inspected here (slot 2) http://lcgapp.cern.ch/spi/cgi-bin/nightly.py

Build Gaudi and Root6

To build Gaudi on pcphsft60, it is necessary to use the cmt infrastructure. In order to build Gaudi on top of root located

cd /build/gaudi_with_cmt
source setup.sh
cd LCGCMT/LCG_Builders/GAUDI/cmt
cmt config
source setup.sh
rm -r /build/gaudi_with_cmt/work/GAUDI/GAUDI_no_reflex
cmt pkg_get
cmt pkg_config
cmt pkg_make

and for rebuilding:

cmt pkg_clean
rm -r /build/gaudi_with_cmt/work/GAUDI/GAUDI_no_reflex
cmt pkg_get
cmt pkg_config
cmt pkg_make

In order to obtain a tuned gaudi no reflex version:

git clone /afs/cern.ch/work/d/dpiparo/GitRepos/GAUDI_no_reflex.git/
cd GAUDI_no_reflex
git checkout spi/GAUDI_no_reflex_tuned

Tuned are the changes introduced starting from the version Marco set up.

Genreflex calls in Gaudi

There are 11 calls to genreflex in Gaudi, none crashes. Their list is the following:

Case 1:

/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../ROOT/ROOT_today/x86_64-slc6-gcc48-opt/root/bin/genreflex 
/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiKernel/dict/dictionary.h 
-o ../x86_64-slc6-gcc48-opt/dict/GaudiKernel/dictionary_dict.cpp  
--gccxmlopt='--gccxml-compiler lcg-g++-4.8.1 ' 
--select=/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiKernel/dict/dictionary.xml  
--gccxmlpath=/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/gccxml/0.9.0_20120309p2/x86_64-slc6-gcc48-opt/bin 
--rootmap=GaudiKernelDict.rootmap 
--rootmap-lib=libGaudiKernelDict  
-D_GNU_SOURCE 
-DGAUDI_V20_COMPAT  
-DBOOST_FILESYSTEM_VERSION=3  
-I"/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiKernel"  
-I"/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/InstallArea/x86_64-slc6-gcc48-opt/include" 
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../ROOT/ROOT_today/x86_64-slc6-gcc48-opt/root/include"
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/Boost/1.50.0_python2.7/x86_64-slc6-gcc48-opt/include/boost-1_50" 
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Platforms/src"

• Error: Bad tag. The comments with < or > are not properly treated in the selection file. This has been fixed pon Mon Aug the 5th [[][commit]].

Case 2

bin/genreflex 
/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiGSL/dict/GaudiGSLMath.h 
-o ../x86_64-slc6-gcc48-opt/dict/GaudiGSLMath/GaudiGSLMath_dict.cpp 
--split 
--reflex 
--gccxmlopt='--gccxml-compiler lcg-g++-4.8.1 ' 
--select=/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiGSL/dict/GaudiGSLMath.xml  
--gccxmlpath=/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/gccxml/0.9.0_20120309p2/x86_64-slc6-gcc48-opt/bin
--split 
--reflex 
--gccxmlopt='--gccxml-compiler lcg-g++-4.8.1 ' 
--select=/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiGSL/dict/GaudiGSLMath.xml  
--gccxmlpath=/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/gccxml/0.9.0_20120309p2/x86_64-slc6-gcc48-opt/bin --rootmap=GaudiGSLMathDict.rootmap 
--rootmap-lib=libGaudiGSLMathDict  
-D_GNU_SOURCE -DGAUDI_V20_COMPAT  
-DBOOST_FILESYSTEM_VERSION=3  
-I"/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiGSL"  
-I"/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/InstallArea/x86_64-slc6-gcc48-opt/include" 
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../ROOT/ROOT_today/x86_64-slc6-gcc48-opt/root/include" 
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../ROOT/ROOT_today/x86_64-slc6-gcc48-opt/root/include"
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/Boost/1.50.0_python2.7/x86_64-slc6-gcc48-opt/include/boost-1_50" -I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/xrootd/3.2.7/x86_64-slc6-gcc48-opt/include" 
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/GSL/1.10/x86_64-slc6-gcc48-opt/include"
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/clhep/1.9.4.7/x86_64-slc6-gcc48-opt/include" 
-I"/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Platforms/src"                               

• Error: The rootmap which is generated is not found, even if the name seems correct. The reason is that the rootmap is not put where the header is.
• The obsolete (also for root5 it has no effect!) option --reflex is used.
• The option --split is used and is not implemented in the new genreflex. It is used to manage in a special way the content of header and implementation of dictionaries, for example for the specialisation of templated functions before the declaration of stubs. It should not be needed with the new infrastructure.
• there is a repetition of

  --split 
  --reflex 
  --gccxmlopt='--gccxml-compiler lcg-g++-4.8.1 ' 
  --select=/build/nightlies/dev2/Mon/GAUDI/GAUDI_no_reflex/src/GaudiGSL/dict/GaudiGSLMath.xml  
  --gccxmlpath=/build/nightlies/dev2/Mon/LCGCMT/LCGCMT_root6/LCG_Settings/../../../../../experimental/gccxml/0.9.0_20120309p2/x86_64-slc6-gcc48-opt/bin 

The above bugs were fixed. Another existing problem is present with selections of the type

  <method pattern="=*"/>

Selection rules

The user can steer the process of dictionary generation using a selection file (also called selection xml or selection rules). This is passed to genreflex with the switches -s ( --selection_file=). In this file, rules can be set in order to for example exclude symbols from the dictionary such as class members of methods. Typically, a selection file looks like this:

          <lcgdict>
           [<selection>]
             <class [name="classname"] [pattern="wildname"] 
                    [file_name="filename"] [file_pattern="wildname"] 
                    [id="xxxx"] [type="vector"]/>
             <class name="classname" >
               <field name="m_transient" transient="true"/>
               <field name="m_anothertransient" transient="true"/>
               <properties prop1="value1" [prop2="value2"]/>
             </class>
             <function [name="funcname"] [pattern="wildname"] />
             <enum [name="enumname"] [pattern="wildname"] />
             <variable [name="varname"] [pattern="wildname"] />
           [</selection>]
           <exclusion>
             <class [name="classname"] [pattern="wildname"] />
               <method name="unwanted" />
             </class>
           ...
           </lcgdict>

Standard XML comments are available.

Accepted tags

This is the list of tags which are usable in a selection file:

• class
• struct
• function
• variable
• enum
• method
• field
• lcgdict
• selection
• exclusion
• properties

Example selection rules from LHCb software stack

This is a list of examples of selections coming from the LHCb software stack.

•  <class name = "GaudiMP::PyROOTPickle"/> 

•  <class pattern="KeyedContainer<*>"/>

•  <class name="std::vector<MyTrack*>"/>

•  <class name="LHCb::RawEvent" id="000003EA-0000-0000-0000-000000000000">
         <field name="m_eventMap" transient="true"/>
         <field name="m_mapped" transient="true"/>
      </class> 

•  <class name="std::map<LHCb::RawBank::BankType, std::vector<LHCb::RawBank*> >"/>

•  <class name="LHCb::ODIN" id="000003ed-0000-0000-0000-000000000000"/>

•  <class name="SmartRef<LHCb::ODIN>"> <field name="m_target" transient="true"/> </class>

•  <class name="HepMC::GenVertex::particles_in_const_iterator" />

•  <class name="std::map<std::string,HepMC::WeightContainer::size_type>" />

  (Said not to be working with gcc4.6)

•  <class pattern = "*HltDecReport*">

•  <variable pattern="Rich::NRiches"  />

•  <enum name = "Rich::DetectorType" />

•  <function pattern="Rich::detectors*" />

It is curious to note that In the whole set of selection files of LHCb, not a single selection tag was found, it was always omitted (indeed it's optional). The full list of selection files of LHCb can be found here (~300 lines).

Wild card matching

The wildcards allowed in the selection rules are shell-like wildcards and not regular expressions. The implementation in root5 relies on the python fnmatch module. You can inspect the code here. For the root6 version of genreflex and related classes (all in C++), we can rely on the following C functions:

• Posix: fnmatch
• Windows: PathMatchSpec

Declaration annotation for the I/O

The new reflection system does not foresee at the moment exclusion/inclusion of classes, methods, functions and members from the interpreter. Everything will be digested. The important point is to properly process the selection rules for IO, for example to decide which classes are selected for persistence, which ones are excluded and which ones have transient members. The information will be injected into the AST as declaration annotations, using the void clang::Decl::addAttr(Attr* A) method of the Decl class (see here). The actual line is contained in the DecorateDecl function in the rootcling executable. The attribute to be created is an AnnotateAttr instance http://clang.llvm.org/doxygen/classclang_1_1AnnotateAttr.html. The documentation online is not corresponding to the new signature though, which is AnnotateAttr (SourceRange sr, ASTContext &C, llvm::StringRef ann). Genreflex should be able to create such annotations in order that the I/O system can consider them. The attributes to be present in the annotation identified up to now are:

• transient
• id

Selection and Exclusion

The two main parts of the selection file. Selection can also not be explicit, while exclusion must. The user can therefore decide what should go and what should not go in the dictionary. In absence of a selection file, all the symbols are assumed to be selected.

Testing Gaudi against ROOT6

Genconf causes the Gaudi build to crash. The stacktrace is the following:

Program received signal SIGSEGV, Segmentation fault.
0x0000003870932d9f in __strlen_sse42 () from /lib64/libc.so.6
(gdb) bt
#0  0x0000003870932d9f in __strlen_sse42 () from /lib64/libc.so.6
#1  0x00007ffff4a0b95c in ROOT::TMetaUtils::GetModuleFileName(char const*) () from /build/dani/RootDevel/Root6/root/lib/libCling.so
#2  0x00007ffff498d330 in TCling::RegisterModule(char const*, char const**, char const**, char const**, char const**, char const**, void (*)()) ()
    at /build/dani/RootDevel/Root6/root/core/meta/src/TCling.cxx:1055
#3  0x00007ffff62cbd9d in TROOT::InitInterpreter() () at /build/dani/RootDevel/Root6/root/core/base/src/TROOT.cxx:1594
#4  0x00007ffff62c6480 in ROOT::GetROOT2() () at /build/dani/RootDevel/Root6/root/core/base/src/TROOT.cxx:295
#5  0x00007ffff62c6499 in ROOT::GetROOT() () at /build/dani/RootDevel/Root6/root/core/base/src/TROOT.cxx:302
#6  0x00007ffff6332ee3 in ROOT::TGenericClassInfo::~TGenericClassInfo() () from /build/dani/RootDevel/Root6/root/lib/libCore.so
#7  0x0000003870835de2 in exit () from /lib64/libc.so.6
#8  0x000000387081ece4 in __libc_start_main () from /lib64/libc.so.6
#9  0x0000000000416061 in _start ()

This behaviour is not present with a root build performed with clang, only with gcc. A jira bug was filed here.