InhomCosmIIISoftware < Cosmo

<< Cosmo.InhomCosmIII

%TOC%

----



---+ Inhomogeneous Cosmologies III Software notes

---++ See also:
   * *scalar averaging* software - ramses-scalav: CosmoTorun17Software#workshop_notes_45_scalar_averaging_N_45body
   * *SageManifolds* - CosmoTorun17Software#workshop_notes_45_Sage_Manifolds

---+ Einstein Toolkit tutorial

   * *main URL 2017: http://cosmo.torun.pl/cosmotorun17_soft.html* + notes at CosmoTorun17Software#workshop_notes_45_ET

---++ Workshop notes - ET
   * FIRST download:
      * as per instructions on: http://einsteintoolkit.org
      * or *easier*: Please try to install the ET (and perhaps reproduce a simple spacetime) prior to the workshop, following the instructions at https://docs.einsteintoolkit.org/et-docs/Simplified_Tutorial_for_New_Users.

---++ Download FLRWSolver and add it to the ET
   * Clone the public FLRWSolver version from bitbucket using: =git clone https://github.com/hayleyjm/FLRWSolver_public=
      * %RED%Warning: do not use the 'gh' command recommended by Github, since it aims at [[https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish][vendor lockin]], also known as "destroying your academic freedom".%ENDCOLOR%
   * Rename the folder =flrwsolver=, and move to =Cactus/arrangements/EinsteinInitialData/=
   * Edit =thornlists/einsteintoolkit.th= in your top Cactus directory
   * Find =EinsteinInitialData=
   * Add the line: =EinsteinInitialData/FLRWSolver= beneath other thorns
   * You may safely comment out the line =Llama/WaveExtractL= since it is heavy (RAM/cpu time) to compile and not needed.
   * You may also safely comment out =EinsteinInitialData/NoExcision=, or any of the =Meudon_BH= thorns (if you run into errors)
   * Compile: =./simfactory/bin/sim build --mdbkey make 'make -j2' --thornlist=thornlists/einsteintoolkit.th --optionlist=myoptionlist.cfg=
      * where =myoptionlist.cfg= is the appropriate optionlist for your machine -- look in =simfactory/mdb/optionlists for a list=
   * (You will probably first be asked to use =sim setup= ; you should create a directory where you want to store simulations, and then run =./simfactory/bin/sim setup= ; for local usage, you can accept the defaults for the "key" values.)
   * Run: =./simfactory/bin/sim submit FLRW1 --parfile=path/to/FLRW.par --procs=1 --walltime=0:30:0=
   * OR: navigate to your simulations/ directory and simply run: =./cactus_sim FLRW.par= (where =cactus_sim= is the executable, located in =Cactus/exe/=. Create a softlink in your simulations directory with =ln -s /path/to/Cactus/exe/cactus_sim .= )

---++ Checking your solutions
   * You will also find a simple Python script =exactsolns.py= in the FLRWSolver repository to check how your simulation is working.
   * Check the script, and make sure the indicated parameters match your simulation
   * This will generate some PDF plots which you can see how well you match the FLRW solution
   * If you get some weird-looking solutions (e.g. density dropping to very low values suddenly), do not fear, try increasing the parameter =EOS_Omni::poly_k= in your .par file and re-running the simulation (to a *max* value of 0.1)

---++ More interesting runs
   * We can repeat the above run using instead =FLRW_perturb.par= to simulate a simple single-mode perturbation to a background FLRW spacetime
   * Once this is done, the same python script =exactsolns.py= can be edited to instead compare solutions for the growth of linear perturbations (set =FLRW=False=)
   * Note that the perturbations may not match the exact solution very well -- this is simply because we are currently running a 10^3 grid, so increase the resolution (change dx,dy,dz in your parameter file) and you will see a better match!
   * We can use =SPLASH= to visualise our 3D HDF5 data output from Cactus.

---++ SPLASH
   * Usually used for visualising particle distributions
   * Daniel has modified it to read in Cactus HDF5 data
   * We first need to modify the Cactus output to have one file (with all variables included) per timestep. To do this, run the Python script (also located in the FLRWSolver repository): =python split_HDF5_per_iteration.py=
   * This is written using Python2.7, and you will need to install =h5py= and =numpy=
   * Once you have run this, you will have some files named =FLRW_perturb_it0.hdf5, FLRW_perturb_it10.hdf5=... and so on.
   * These files can then be visualised using =SPLASH=. Download from: =http://users.monash.edu.au/~dprice/splash/= and first compile as per instructions on the webpage
   * We then need to compile the Cactus capabilities separately: =make SYSTEM=gfortran cactus= 
   * You should now be able to visualise your data with: =csplash-hdf5 FLRW_perturb_it0.hdf5= and select from the list which variable you want to look at!
   * Enjoy!

---++ Hints
   * *avoid filling home directory:* To avoid creating a huge (733 Mb) directory in your home directory for the simplified tutorial for new users, create your target directory and soft link it to =~/simulations= either before you start or before you start running a simulation.

---+++ Successful runs of tutorial
   * CosmoTorun17: Krzysztof, Boude, Sven
   * InhomCosmIII: Quentin

---+ Gevolution hackathon
   * gevolution:
      * https://github.com/gevolution-code/gevolution-1.1
      * ArXiv:1604.06065
      * makefile edit for Debian GNU/Linux (stretch): <verbatim>INCLUDE += -I../LATfield2 -I/usr/include/hdf5/serial -L/usr/lib/x86_64-linux-gnu/hdf5/serial</verbatim> or <verbatim>INCLUDE += -I../LATfield2 -I/usr/include/hdf5/openmpi -L/usr/lib/x86_64-linux-gnu/hdf5/openmpi</verbatim>
      * LATfield2 hack: see [[https://github.com/broukema/LATfield2/commit/4479e1c29c353990e8e5a3582c4fd611f3d9b427][this hdf5.h hack]] to [[https://github.com/daverio/LATfield2][daverio LATfield2]]
      * if you unload both package directories at the same directory level, then in =gevolution-1.1/= you can do =ln -s ../LATfield2 .= so that the gevolution makefile can find the LATfield2 files
      * Create an output directory =mkdir output/=
      * example libraries you will need: =sudo aptitude install libfftw3-dev openmpi-bin libopenmpi-dev libhdf5-openmpi-dev libhdf5-dev hdf5-helpers=
      * You should be able to compile gevolution with =make= and run the example with <verbatim>(time mpirun -np 4 ./gevolution -n 2 -m 2 -s settings.ini) > gevolution.log 2>&1</verbatim> in order to get a nice log file - on Boud's 4-core laptop this took about 140 seconds of user time and created many output files in =output/= .
   * Visualisation
      * SPLASH:
         * http://users.monash.edu.au/~dprice/splash/
         * Debian GNU/Linux derivatives (Ubuntu, Mint, ...): _aptitude update && aptitude install splash_
            * DebianBackportSplash
      * gadgetviewer (GPLv3): https://github.com/jchelly/gadgetviewer

---++ Hints
   * To test run on a machine with less than four cores, a hack to try is: _mpirun -np 4 --map-by core:OVERSUBSCRIBE"_ ; you might get a segmentation fault at the end of the run, after output files are created

---+ Maxima
   * debian derivatives: _aptitude install maxima_
   * http://maxima.sourceforge.net
   * Szymon - general introduction - =git clone https://bitbucket.org/broukema/maxima_flrw=
   * Boud - example of using ctensor: FLRWEquationsGR
   * [[https://www.springer.com/gp/livingreviews/relativity/news/living-reviews-relativity-computer-algebra-in-gravity-research/16056832][Malcolm !MacCallum Living Review on Relativity on computer algebra systems]]

---+ Ray-tracing: gyoto
   * TODO: learn https://gyoto.obspm.fr for ray-tracing independent of ET and gevolution

-- Main.BoudRoukema - 13 Sep 2018 + ...
   * gevolution phi P(k) example: <br />
     <img src="%ATTACHURLPATH%/gevolution_phi_test.png" alt="gevolution_phi_test.png" width='570' height='570' />
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
png	gevolution_phi_test.png	manage	2.1 K	18 Sep 2018 - 15:23	BoudRoukema	gevolution phi P(k) example
This topic: Cosmo > WebPreferences > WebHome > CosmoConferences > CosmoTorun17Software > InhomCosmIIISoftware
Topic revision: 12 Jul 2021, BoudRoukema (raw view)
Copyright © CC-BY-SA by the contributing authors. All material on this collaboration platform is copyrighted under CC-BY-SA by the contributing authors unless otherwise noted.
Ideas, requests, problems regarding Foswiki? Send feedback