Advanced Usage (Script)

After the installation, the next sections describe how to get started. The sections configuration and logging describe the general methods we use, this is helpful to understand how you can change model parameters and similar.

Advanced Installation (Linux)

This is a quick guide on how to install AvaFrame and the required dependencies on your machine. AvaFrame is developed on Linux machines (Ubuntu/Manjaro/Arch) with recent Python versions > 3.8. These instructions assume you are familiar with working in a terminal. This guide is described for Linux. For Windows, see Advanced Installation (Windows).

Requirements

Install git, python and pixi. Some operating systems might require the python headers (e.g python-dev on ubuntu) or other supporting libraries/packages (e.g. Visual Studio on Windows needs the c++ compiler components).

Setup AvaFrame

Clone the AvaFrame repository (in a directory of your choice: [YOURDIR]) and change into it:

cd [YOURDIR]
git clone https://github.com/avaframe/AvaFrame.git
cd AvaFrame

Run pixi:

pixi shell

Compile the cython com1DFA part. You might also have to install a c-compiler (gcc or similar) through your systems package manager:

python setup.py build_ext --inplace

Warning

You will have to do this compilation every time something changes in the cython code. We also suggest to do this everytime updates from the repositories are pulled.

All this installs avaframe in editable mode, so every time you import avaframe the current (local) version will be used.

If you want to have the lastet stable release instead, run:

pixi shell --environment prod

Test it by starting python and do an import avaframe. If no error comes up, you are good to go.

To see the current version, you can use:

pixi list avaframe

Head over to First run for the next steps.

Advanced Installation (Windows)

This is a quick guide on how to install AvaFrame and the required dependencies on your machine. AvaFrame is developed on Linux machines (Ubuntu/Manjaro/Arch) with recent Python versions > 3.8. These instructions assume you are familiar with working in a terminal. This guide is described for Windows. For Linux, see Advanced Installation (Linux).

Requirements

Install git, python and pixi.

Install Microsoft C++ compiler. Follow the installation steps for the version corresponding to the installed python version.

Setup AvaFrame

Clone the AvaFrame repository (in a directory of your choice: [YOURDIR]) and change into it:

cd [YOURDIR]
git clone https://github.com/avaframe/AvaFrame.git
cd AvaFrame

Run pixi:

pixi shell

Note

If you get some access denied error in Powershell, you might need to run the command

Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser

Compile the cython com1DFA part:

python setup.py build_ext --inplace

Warning

You will have to do this compilation every time something changes in the cython code. We also suggest to do this everytime updates from the repositories are pulled.

Before compilation in Windows, make sure to delete AvaFrame/build directory, in addition to any .pyd, .c, and .pycache files in AvaFrame/avaframe/com1DFA

This installs avaframe in editable mode, so every time you import avaframe the current (local) version will be used.

If you want to have the lastet stable release instead, run:

pixi shell --environment prod

Test it by starting python and do an import avaframe. If no error comes up, you are good to go.

Head over to First run for the next steps.

Using QGIS from with an advanced installation

Assuming you followed the steps above, you should get a working QGis installation by running (it will take a while):

pixi shell --environment qgis

An to run QGIS:

qgis

Depending on your mode of QGis installation (direct installer, conda, OSGeo4W…) your script AvaFrame installation might be overruled…

Now you can install the AvaFrameConnector plugin via QGIS as per usual (Operational Installation).

First run

Follow these steps to run your first simulation (assuming you finished the advanced installation steps; if you come back later, make sure to run pixi shell again to activate the environment):

change into your AvaFrame directory (replace [YOURDIR] with your path from the installation steps):
```
cd [YOURDIR]/AvaFrame/avaframe
```
run:
```
python runCom1DFA.py
```

a similar output should show up:

logUtils - INFO -  Started logging at: 03.11.2020 22:42:04
logUtils - INFO -  Also logging to: data/avaParabola/runCom1DFA.log
runCom1DFA - INFO -  MAIN SCRIPT
runCom1DFA - INFO -  Current avalanche: data/avaParabola
...

This will perform a dense flow avalanche simulation using the com1DFA module. The results are saved to data/avaParabola/Outputs/com1DFA. For a first look at the results, got to the folder reports, there you can find a markdown report of the simulations performed including some plots of the results.

To display markdown files in a nice way use a markdown viewer of your choice. Some other options are:

Use the Atom editor with a markdown plugin
If you have pandoc installed use this to convert it to pdf/html
Some browsers have markdown extensions you can install easily

Workflow example

The following example should make it easier for you to find your way in AvaFrame and setup your own AvaFrame workflow after you did the full setup. There is also a directory with examples for different workflows, see more here: Example runscripts.

Make sure you change to your AvaFrame directory by:

cd [YOURDIR]/AvaFrame

Replace [YOURDIR] with the directory from your installation step.

Initialize project

To create the folder where the input data lies and where the output results will be saved, specify the full path to the folder in the local_avaframeCfg.ini (which is a copy of avaframeCfg.ini that you need to create). So:

cd avaframe
cp avaframeCfg.ini local_avaframeCfg.ini

and edit local_avaframeCfg.ini with your favorite text editor and adjust the variable avalancheDir.

Then run

python runScripts/runInitializeProject.py

This will create a new directory with the input required by AvaFrame structured as described in Initialize Project.

Input data

Check the input data required by the different modules you want to use and fill the Inputs/ inside the [avalancheDir] folder from the initialize step accordingly.

For example the com1DFA module needs input as described in Input. You can also have a look at the default setting for the module you want to use (for example com1DFACfg.ini for module com1DFA). If you want to use different settings, create a local_ copy of the .ini file and modify the desired parameters.

More information about the configuration can be found here: Configuration

Building your run script

Create your own workflow by taking the runOperational.py script as template.

We suggest you copy it and adjust it to your liking. There are annotations in the code that should help you to understand the structure.

A lot more examples can be found in the runScripts directory (see also Example runscripts).

Configuration

In order to set the configurations required by all the modules within Avaframe, the python module configparser is used.

This is done in two steps. The first step fetches the main settings:

from avaframe.in3Utils import cfgUtils
# Load avalanche directory from general configuration file
cfgMain = cfgUtils.getGeneralConfig()
avalancheDir = cfgMain['MAIN']['avalancheDir']

In the second step the specific settings to a given module are imported:

from avaframe.tmp1Ex import tmp1Ex
# Load all input Parameters from config file
# get the configuration of an already imported module
# Write config to log file
cfg = cfgUtils.getModuleConfig(tmp1Ex)

The in3Utils.cfgUtils.getModuleConfig() function reads the settings from a configuration file (tmpEx.ini in our example) and writes these settings to the log file. The default settings can be found in the configuration file provided within each module.

It is possible to modify these settings, there are two options:

provide the path to your own configuration file when calling cfgUtils.getModuleConfig(moduleName, path to config file)
create a copy of the module configuration file called local_ followed by the name of the original configuration file and set the desired values of the individual parameters.
see Override configuration for additional options to modify configuration

So the order is as follows:

if there is a path provided, configuration is read from this file.
if there is no path provided, the local_... configuration file is read if it exists.
if there is no local_..., the getModuleConfig function reads the settings from the default configuration file with the default settings.

In the configuration file itself, there are multiple options to vary a parameter:

replace the default parameter value with desired value
provide a number of parameter values separated by | (e.g. relTh=1.|2.|3.)
provide a number of parameter values using start:stop:numberOfSteps (e.g. relTh=1.:3.:3)) - a single value can be added by appending &4.0 for example

Override configuration

If tools of one module, let’s call this module A for now, are called from another module B, there is the option to include an collectionName_A_override section in the configuration file of module B. In this case, the default configuration of module A is read and the parameters in the A_override section in the module B configuration file are used to update the configuration settings of module A. This has the advantage of gathering all the configuration parameters used for one task in one configuration file. An example of this usage can be found in ana1Tests/energyLineTestCfg.ini.

Logging

In order to generate simulation logs and to control what is prompted to the terminal, we use the python module logging.

Let’s have a look at the simple example in runScripts.runTmp1Ex and tmp1Ex.tmp1Ex() on how this is used within AvaFrame.

In your main script call:

from avaframe.in3Utils import logUtils
# log file name; leave empty to use default runLog.log
logName = 'runTmp1Ex'
# specify the working directory
avalancheDir = './'
# ---------------------------------------------
# Start logging
log = logUtils.initiateLogger(avalancheDir, logName)

This will configure the logging (it sets the console output as well as the log file). In your modules/subscripts add:

import logging
log = logging.getLogger(__name__)

So you can use:

log.debug('Should be here')
log.info('DEM : %s',variable)

To get output that looks like this in your console:

tmp1Ex:DEBUG - Should be here
tmp1Ex:INFO - DEM : /path/to/DEM

and something similar in the .log file which is saved in ./runTmp1Ex.log in this example. The logging configuration is set in AvaFrame/avaframe/in3Utils/logging.conf.

You can modify this logging.conf file to modify the levels or format of the messages to display (python doc will help you).

Example runscripts

In runScripts we provide ready-to-use scripts for different applications of the modules provided within AvaFrame.

Derive input data

runScripts.runComputeDist

Create a new project

runScripts.runInitializeProject

Generate idealized/generic topography data

runScripts.runGenerateTopo
runScripts.runGenProjTopoRelease

Postprocessing

runScripts.runAna3AIMEC
runScripts.runAna3AIMECCompMods
runScripts.runAna3AIMECIndi
runScripts.runStatsExample
runScripts.runProbAna

Visualisation

runScripts.runQuickPlotSimple
runScripts.runQuickPlotOne
runScripts.runPlotTopo
runScripts.runExportToCsv

Testing

runScripts.runDamBreak
runScripts.runSimilaritySol
runScripts.runTestFP
runScripts.runStandardTestsCom1DFAOrig
runScripts.runComparisonModules
runScripts.runFetchBench
runScripts.runWriteDesDict

Update AvaFrame

To update go to your avaframe repository [YOURDIR]/Avaframe, pull the latest changes via:

git pull

and repeat the compilation step from above:

python setup.py build_ext --inplace

If there are updates on the requirements inside setup.py, it might be necessary to run:

pip install -e .

again to get the additional requirements installed.