Commit de7ac041 authored by Mark Olesen's avatar Mark Olesen

COMP: adjustments to makeCGAL for compilation with alternative mpfr/gmp

- document special steps for using ThirdParty mpfr/gmp with clang etc
parent 7419702a
...@@ -83,9 +83,9 @@ automatically as part of the top-level OpenFOAM `Allwmake`. ...@@ -83,9 +83,9 @@ automatically as part of the top-level OpenFOAM `Allwmake`.
Nonetheless it may be necessary or useful to build various Nonetheless it may be necessary or useful to build various
ThirdParty components prior to building OpenFOAM itself. ThirdParty components prior to building OpenFOAM itself.
### Bootstrapping *(optional)*
### Build Sequence * `makeGcc` _or_ `makeLLVM` <a name="makeGcc"></a>
1. `makeGcc` _or_ `makeLLVM` <a name="makeGcc"></a> *(optional)*
- Makes a third-party [gcc](#gcc-compiler) or [clang](#clang-compiler) installation, - Makes a third-party [gcc](#gcc-compiler) or [clang](#clang-compiler) installation,
which is needed if the system gcc is [too old](#gcc-compiler). which is needed if the system gcc is [too old](#gcc-compiler).
If your system compiler is recent enough, you can skip this step. If your system compiler is recent enough, you can skip this step.
...@@ -93,30 +93,75 @@ ThirdParty components prior to building OpenFOAM itself. ...@@ -93,30 +93,75 @@ ThirdParty components prior to building OpenFOAM itself.
OpenFOAM `etc/bashrc` or your equivalent `prefs.sh` file: OpenFOAM `etc/bashrc` or your equivalent `prefs.sh` file:
- `WM_COMPILER_TYPE=ThirdParty` - `WM_COMPILER_TYPE=ThirdParty`
- `WM_COMPILER=Gcc48` (for example) - `WM_COMPILER=Gcc48` (for example)
- or `WM_COMPILER=Clang` and adjust the `clang_version` entry in the OpenFOAM - `WM_COMPILER=Clang40` (for example)
- or `WM_COMPILER=Clang` and adjust `clang_version` in the OpenFOAM
`etc/config.sh/compiler` or equivalent. `etc/config.sh/compiler` or equivalent.
- More description is contained in the header comments of the - More description is contained in the header comments of the
`makeGcc` and `makeLLVM` files. `makeGcc` and `makeLLVM` files.
- *Attention*: If you are building a newer version of clang, you may need to - *Attention*: If you are building a newer version of clang, you may need to
update your CMake beforehand. update your CMake beforehand.
2. `makeCmake` *(optional)* * `makeCmake`
- Makes a third-party [CMake](#general-packages) installation, which is - Makes a third-party [CMake](#general-packages) installation, which is
needed if a system CMake does not exist or is [too old](#min-cmake), needed if a system CMake does not exist or is [too old](#min-cmake),
- Note that CMake is being used by an number of third-party packages - Note that CMake is being used by an number of third-party packages
(CGAL, LLVM, ParaView, VTK, ...) (CGAL, LLVM, ParaView, VTK, ...)
so this may become an increasingly important aspect of the build. so this may become an increasingly important aspect of the build.
3. `Allwmake`
Note that the order of the bootstrapping process may need to be
reversed, or even require a few loops. For example, if you may need a
newer version of CMake before being able to build LLVM/Clang and
subsequently use the newly build clang to create a newer version of
CMake in the desired location.
Additionally, if you are using clang but with ThirdParty locations for
gmp/mpfr you will need some extra work. Here is an example:
* Compile a new ThirdParty clang version:
./makeLLVM llvm-4.0.1
* Now adjust the OpenFOAM `prefs.sh` to use the new compiler settings,
and update the OpenFOAM environment (eg, `wmRefresh`)
* Next use (abuse) the `makeGcc` script to compile gmp/mpfr libraries.
It is best to pass the desired versions explicitly, and necessary
to set the CC/CXX variables so that the correct compiler is used:
CC=clang CXX=clang++ ./makeGcc gmp-6.1.2 mpfr-4.0.0 gcc-system
specifying `gcc-system` effectively disables building of gcc,
but will build the gmp/mpfr components.
* As a final step, it will be necessary to add the ThirdParty
gmp/mpfr locations in the OpenFOAM config files since they are
normally only used in combination with a ThirdParty gcc.
The location to make these changes is in the `etc/config.sh/CGAL`,
since this is the component that uses the mpfr library.
For example,
gmp_version=gmp-6.1.2
mpfr_version=mpfr-4.0.0
export GMP_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$gmp_version
export GMP_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$mpfr_version
* Update update the OpenFOAM environment (eg, `wmRefresh`) again.
### Build Sequence
1. `Allwmake`
- This will be automatically invoked by the top-level OpenFOAM `Allwmake`, but - This will be automatically invoked by the top-level OpenFOAM `Allwmake`, but
can also be invoked directly to find possible build errors. can also be invoked directly to find possible build errors.
- Builds an mpi library (openmpi or mpich), scotch decomposition, boost, CGAL, FFTW. - Builds an mpi library (openmpi or mpich), scotch decomposition, boost, CGAL, FFTW.
- If the optional kahip or metis directories are found, they will also be compiled. - If the optional kahip or metis directories are found, they will also be compiled.
4. `makeParaView` *(optional but highly recommended)* 2. `makeParaView` *(optional but highly recommended)*
- This is optional, but extremely useful for visualization and for - This is optional, but extremely useful for visualization and for
run-time post-processing function objects. run-time post-processing function objects.
You can build this at a later point in time, but then you should You can build this at a later point in time, but then you should
remember to rebuild the post-processing function objects and the remember to rebuild the post-processing function objects and the
reader module as well. reader module as well.
5. Make any additional optional components 3. Make any additional optional components
#### Optional Components #### Optional Components
...@@ -180,6 +225,9 @@ and save some disk space. ...@@ -180,6 +225,9 @@ and save some disk space.
## Build Notes ## Build Notes
### CGAL
- The zlib library and zlib development headers are required.
### Scotch ### Scotch
- The zlib library and zlib development headers are required. - The zlib library and zlib development headers are required.
...@@ -480,4 +528,4 @@ that clang compiler for building the newer llvm/clang version. ...@@ -480,4 +528,4 @@ that clang compiler for building the newer llvm/clang version.
--- ---
Copyright 2016-2017 OpenCFD Ltd Copyright 2016-2018 OpenCFD Ltd
...@@ -60,11 +60,10 @@ cd ${0%/*} && wmakeCheckPwd "$WM_THIRD_PARTY_DIR" 2>/dev/null || { ...@@ -60,11 +60,10 @@ cd ${0%/*} && wmakeCheckPwd "$WM_THIRD_PARTY_DIR" 2>/dev/null || {
} }
. etc/tools/ThirdPartyFunctions . etc/tools/ThirdPartyFunctions
#------------------------------------------------------------------------------ #------------------------------------------------------------------------------
unset BOOST_ARCH_PATH CGAL_ARCH_PATH GMP_ARCH_PATH MPFR_ARCH_PATH # Purge old
unset BOOST_ARCH_PATH CGAL_ARCH_PATH # Purge current values
# CGAL, boost and gmp/mpfr versions from OpenFOAM etc/config.sh files. # CGAL, boost and gmp/mpfr versions from OpenFOAM etc/config.sh files.
# Get compiler first and let CGAL config override GMP (eg, when there is no GMP) # Get compiler first and let CGAL config override GMP and MPFR
_foamEtc config.sh/compiler _foamEtc config.sh/compiler
_foamEtc config.sh/CGAL _foamEtc config.sh/CGAL
...@@ -124,24 +123,26 @@ do ...@@ -124,24 +123,26 @@ do
gmpPACKAGE="gmp-system" gmpPACKAGE="gmp-system"
mpfrPACKAGE="mpfr-system" mpfrPACKAGE="mpfr-system"
boostPACKAGE="boost-system" boostPACKAGE="boost-system"
unset BOOST_ARCH_PATH # avoid inconsistency unset BOOST_ARCH_PATH GMP_ARCH_PATH MPFR_ARCH_PATH
;; ;;
-no-lib) -no-lib)
optHeadersOnly=true optHeadersOnly=true
;; ;;
gmp-[4-9]* | gmp-system | gmp-none) gmp-[4-9]* | gmp-system | gmp-none)
gmpPACKAGE="${1%%/}" gmpPACKAGE="${1%%/}"
unset GMP_ARCH_PATH
;; ;;
mpfr-[2-9]* | mpfr-system | mpfr-none) mpfr-[2-9]* | mpfr-system | mpfr-none)
mpfrPACKAGE="${1%%/}" mpfrPACKAGE="${1%%/}"
;; unset MPFR_ARCH_PATH
CGAL-[0-9]*)
cgalPACKAGE="${1%%/}"
unset CGAL_ARCH_PATH # avoid inconsistency
;; ;;
boost-[0-9]* | boost_[0-9]* | boost-system ) boost-[0-9]* | boost_[0-9]* | boost-system )
boostPACKAGE="${1%%/}" boostPACKAGE="${1%%/}"
unset BOOST_ARCH_PATH # avoid inconsistency unset BOOST_ARCH_PATH
;;
CGAL-[0-9]*)
cgalPACKAGE="${1%%/}"
unset CGAL_ARCH_PATH
;; ;;
*) *)
die "unknown option/argument: '$1'" die "unknown option/argument: '$1'"
...@@ -186,14 +187,37 @@ then ...@@ -186,14 +187,37 @@ then
echo "Using boost-system (skip ThirdParty build of BOOST)" echo "Using boost-system (skip ThirdParty build of BOOST)"
# Tagged as 'system' but could actually point to a central location # Tagged as 'system' but could actually point to a central location
[ -d "$BOOST_ARCH_PATH/include" ] || {
if [ -d "$boostInc" ]
then
if BOOST_ARCH_PATH=$(cd $BOOST_ARCH_PATH 2>/dev/null && pwd -P)
then
boostPACKAGE=${BOOST_ARCH_PATH##*/}
else
echo "ERROR: bad path for BOOST_ARCH_PATH"
echo "stopping build"
exit 1
fi
libdir="$BOOST_ARCH_PATH/lib"
else
boostInc="/usr/include" boostInc="/usr/include"
boostLib="/usr/lib$WM_COMPILER_LIB_ARCH" boostLib="/usr/lib$WM_COMPILER_LIB_ARCH"
} libdir="/usr/lib"
fi
# Use lib/ when lib64/ does not exist
[ ! -d "$boostLib" -a -d "$libdir" ] && boostLib="$libdir"
elif [ -f "$boostInc/boost/version.hpp" ] elif [ -f "$boostInc/boost/version.hpp" ]
then then
echo "Using $boostPACKAGE" echo "Using $boostPACKAGE"
libdir="$BOOST_ARCH_PATH/lib"
# Use lib when lib64 does not exist
[ ! -d "$boostLib" -a -d "$libdir" ] && boostLib="$libdir"
else else
echo "Starting build: $boostPACKAGE" echo "Starting build: $boostPACKAGE"
echo echo
...@@ -218,7 +242,7 @@ else ...@@ -218,7 +242,7 @@ else
} }
fi fi
# nothing left to build # Nothing left to build
if _foamIsSystem $cgalPACKAGE if _foamIsSystem $cgalPACKAGE
then then
echo "Using cgal-system (skip ThirdParty build of CGAL)" echo "Using cgal-system (skip ThirdParty build of CGAL)"
...@@ -239,18 +263,6 @@ else ...@@ -239,18 +263,6 @@ else
exit 1 exit 1
fi fi
#------------------------------------------------------------------------------
cat<<SUMMARY
CGAL configuration
------------------
CGAL = $cgalPACKAGE
BOOST = $boostPACKAGE
GMP = $gmpPACKAGE
MPFR = $mpfrPACKAGE
------------------
SUMMARY
#------------------------------------------------------------------------------ #------------------------------------------------------------------------------
# #
# Build CGAL # Build CGAL
...@@ -268,22 +280,61 @@ CGAL_BUILD_DIR=$buildBASE/$cgalPACKAGE ...@@ -268,22 +280,61 @@ CGAL_BUILD_DIR=$buildBASE/$cgalPACKAGE
: ${CGAL_ARCH_PATH:=$installBASE/$cgalPACKAGE} # Fallback : ${CGAL_ARCH_PATH:=$installBASE/$cgalPACKAGE} # Fallback
# gmp/mpfr are installed without compiler name # gmp/mpfr are installed without compiler name
installBASE=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH mpfrBASE=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH
GMP_ARCH_PATH=$installBASE/$gmpPACKAGE
MPFR_ARCH_PATH=$installBASE/$mpfrPACKAGE
# Enable/disable gmp/mpfr together # Enable/disable gmp/mpfr together
if _foamIsNone $gmpPACKAGE || _foamIsNone $mpfrPACKAGE if _foamIsNone $gmpPACKAGE || _foamIsNone $mpfrPACKAGE
then then
GMP_ARCH_PATH=none GMP_ARCH_PATH=none
MPFR_ARCH_PATH=none MPFR_ARCH_PATH=none
elif _foamIsSystem $GMP_ARCH_PATH || _foamIsSystem $MPFR_ARCH_PATH elif _foamIsSystem $gmpPACKAGE || _foamIsSystem $mpfrPACKAGE
then then
GMP_ARCH_PATH=system # for an accurate record # May really be system, but could also by a central installation
MPFR_ARCH_PATH=system # Ensure everything is accurately recorded. Resolve paths etc.
if [ -d "$GMP_ARCH_PATH" ]
then
if GMP_ARCH_PATH=$(cd $GMP_ARCH_PATH 2>/dev/null && pwd -P)
then
gmpPACKAGE=${GMP_ARCH_PATH##*/}
else
echo "ERROR: bad path for GMP_ARCH_PATH"
echo "stopping build"
exit 1
fi
else
GMP_ARCH_PATH=system
fi
if [ -d "$MPFR_ARCH_PATH" ]
then
if MPFR_ARCH_PATH=$(cd $MPFR_ARCH_PATH 2>/dev/null && pwd -P)
then
mpfrPACKAGE=${MPFR_ARCH_PATH##*/}
else
echo "ERROR: bad path for MPFR_ARCH_PATH"
echo "stopping build"
exit 1
fi
else
MPFR_ARCH_PATH=system
fi
else
GMP_ARCH_PATH=$mpfrBASE/$gmpPACKAGE
MPFR_ARCH_PATH=$mpfrBASE/$mpfrPACKAGE
fi fi
#------------------------------------------------------------------------------
cat<<SUMMARY
CGAL configuration
------------------
CGAL = $cgalPACKAGE
BOOST = $boostPACKAGE
GMP = $gmpPACKAGE
MPFR = $mpfrPACKAGE
------------------
SUMMARY
# #
# build information recorded for later use # build information recorded for later use
...@@ -311,7 +362,7 @@ BUILD_INFO ...@@ -311,7 +362,7 @@ BUILD_INFO
} }
# compare expected vs what is extracted as KEY=... in text # Compare expected vs what is extracted as KEY=... in text
# $1 = key # $1 = key
# $2 = expected # $2 = expected
# $3 = text to extract from # $3 = text to extract from
...@@ -327,10 +378,11 @@ infoValueEq() ...@@ -327,10 +378,11 @@ infoValueEq()
fi fi
} }
# needs build # Needs build
cgalIsCurrent() cgalIsCurrent()
{ {
local info=$(cat $buildInfoFile 2>/dev/null) # Only need VAR=... (also avoids embedded '#' that bothers some shells)
local info=$(sed -n -e '/^[A-Z]/p' $buildInfoFile 2>/dev/null)
[ -n "$info" ] || return 1 [ -n "$info" ] || return 1
local libDirName="lib$WM_COMPILER_LIB_ARCH" local libDirName="lib$WM_COMPILER_LIB_ARCH"
...@@ -381,17 +433,20 @@ fi ...@@ -381,17 +433,20 @@ fi
# See http://doc.cgal.org/latest/Manual/installation.html # See http://doc.cgal.org/latest/Manual/installation.html
if _foamIsSystem $boostPACKAGE if _foamIsSystem $boostPACKAGE
then then
echo " system : boost"
# Tagged as 'system' but could actually point to a central location # Tagged as 'system' but could actually point to a central location
if [ -d "$BOOST_ARCH_PATH/include" ] if [ -d "$BOOST_ARCH_PATH/include" ]
then then
echo " boost : ${BOOST_ARCH_PATH##*/}"
configBoost="-DBOOST_ROOT=$BOOST_ARCH_PATH" configBoost="-DBOOST_ROOT=$BOOST_ARCH_PATH"
else
echo " boost : system"
fi fi
## For system - possible that /usr/lib64 not being found? ## For system - possible that /usr/lib64 not being found?
## configBoost="-DBoost_LIBRARY_DIRS=$boostLib" ## configBoost="-DBoost_LIBRARY_DIRS=$boostLib"
elif [ -d "$BOOST_ARCH_PATH" ] elif [ -d "$BOOST_ARCH_PATH" ]
then then
echo " ThirdParty : boost" echo " boost : $boostPACKAGE"
configBoost=$(cat <<CMAKE_OPTIONS configBoost=$(cat <<CMAKE_OPTIONS
-DBoost_INCLUDE_DIR=$boostInc -DBoost_INCLUDE_DIR=$boostInc
-DBoost_LIBRARY_DIRS=$boostLib -DBoost_LIBRARY_DIRS=$boostLib
...@@ -406,44 +461,64 @@ CMAKE_OPTIONS ...@@ -406,44 +461,64 @@ CMAKE_OPTIONS
if _foamIsSystem $GMP_ARCH_PATH if _foamIsSystem $GMP_ARCH_PATH
then then
echo " system : gmp" echo " gmp : system"
elif _foamIsNone $GMP_ARCH_PATH elif _foamIsNone $GMP_ARCH_PATH
then then
echo " disabled : gmp" echo " gmp : disabled"
configGmp="-DCGAL_DISABLE_GMP=TRUE" # Also used for mpfr configGmp="-DCGAL_DISABLE_GMP=TRUE" # Also used for mpfr
elif [ -d "$GMP_ARCH_PATH" ] elif [ -d "$GMP_ARCH_PATH" ]
then then
echo " ThirdParty : gmp" echo " gmp : $gmpPACKAGE"
configGmp=$(cat <<CMAKE_OPTIONS
for libdir in \
$GMP_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH \
$GMP_ARCH_PATH/lib \
;
do
if [ -f "$libdir/libgmp.so" ]
then
configGmp=$(cat <<CMAKE_OPTIONS
-DGMP_INCLUDE_DIR=$GMP_ARCH_PATH/include -DGMP_INCLUDE_DIR=$GMP_ARCH_PATH/include
-DGMP_LIBRARIES_DIR=$GMP_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH -DGMP_LIBRARIES_DIR=$libdir
-DGMP_LIBRARIES=$GMP_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH/libgmp.so -DGMP_LIBRARIES=$libdir/libgmp.so
-DGMPXX_INCLUDE_DIR=$GMP_ARCH_PATH/include
-DGMPXX_LIBRARIES=$GMP_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH/libgmpxx.so
CMAKE_OPTIONS CMAKE_OPTIONS
) )
break
fi
done
else else
echo " system : gmp (did not find $GMP_ARCH_PATH)" echo " system : gmp (did not find $GMP_ARCH_PATH)"
fi fi
if _foamIsSystem $MPFR_ARCH_PATH if _foamIsSystem $MPFR_ARCH_PATH
then then
echo " system : mpfr" echo " mpfr : system"
elif _foamIsNone $MPFR_ARCH_PATH elif _foamIsNone $MPFR_ARCH_PATH
then then
echo " disabled : mpfr" echo " mpfr : disabled"
configGmp="-DCGAL_DISABLE_GMP=TRUE" # Also used for mpfr configGmp="-DCGAL_DISABLE_GMP=TRUE" # Also used for mpfr
elif [ -d "$MPFR_ARCH_PATH" ] elif [ -d "$MPFR_ARCH_PATH" ]
then then
echo " ThirdParty : mpfr" echo " mpfr : $mpfrPACKAGE"
configMpfr=$(cat <<CMAKE_OPTIONS
for libdir in \
$MPFR_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH \
$MPFR_ARCH_PATH/lib \
;
do
if [ -f "$libdir/libmpfr.so" ]
then
configMpfr=$(cat <<CMAKE_OPTIONS
-DMPFR_INCLUDE_DIR=$MPFR_ARCH_PATH/include -DMPFR_INCLUDE_DIR=$MPFR_ARCH_PATH/include
-DMPFR_LIBRARIES_DIR=$MPFR_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH -DMPFR_LIBRARIES_DIR=$libdir
-DMPFR_LIBRARIES=$MPFR_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH/libmpfr.so -DMPFR_LIBRARIES=$libdir/libmpfr.so
CMAKE_OPTIONS CMAKE_OPTIONS
) )
break
fi
done
else else
echo " system : mpfr (did not find $MPFR_ARCH_PATH)" echo " mpfr : system (did not find $MPFR_ARCH_PATH)"
fi fi
cmake=$(findCMake) cmake=$(findCMake)
...@@ -481,7 +556,7 @@ CMAKE_OPTIONS ...@@ -481,7 +556,7 @@ CMAKE_OPTIONS
fi fi
done done
# record our build-status # Record our build-status
recordCGALinfo recordCGALinfo
echo "Done CGAL" echo "Done CGAL"
......
...@@ -39,6 +39,7 @@ cd ${0%/*} && wmakeCheckPwd "$WM_THIRD_PARTY_DIR" 2>/dev/null || { ...@@ -39,6 +39,7 @@ cd ${0%/*} && wmakeCheckPwd "$WM_THIRD_PARTY_DIR" 2>/dev/null || {
} }
. etc/tools/ThirdPartyFunctions . etc/tools/ThirdPartyFunctions
#------------------------------------------------------------------------------ #------------------------------------------------------------------------------
unset GMP_ARCH_PATH MPFR_ARCH_PATH # Purge old
[ "${WM_COMPILER#Gcc}" = "$WM_COMPILER" ] && WM_COMPILER=Gcc # Force gcc [ "${WM_COMPILER#Gcc}" = "$WM_COMPILER" ] && WM_COMPILER=Gcc # Force gcc
WM_COMPILER_TYPE=ThirdParty # Ensure we get the correct settings WM_COMPILER_TYPE=ThirdParty # Ensure we get the correct settings
...@@ -99,12 +100,15 @@ do ...@@ -99,12 +100,15 @@ do
gmpPACKAGE="gmp-system" gmpPACKAGE="gmp-system"
mpfrPACKAGE="mpfr-system" mpfrPACKAGE="mpfr-system"
mpcPACKAGE="mpc-system" mpcPACKAGE="mpc-system"
unset GMP_ARCH_PATH MPFR_ARCH_PATH
;; ;;
gmp-[4-9]* | gmp-system) gmp-[4-9]* | gmp-system)
gmpPACKAGE="${1%%/}" gmpPACKAGE="${1%%/}"
unset GMP_ARCH_PATH
;; ;;
mpfr-[2-9]* | mpfr-system) mpfr-[2-9]* | mpfr-system)
mpfrPACKAGE="${1%%/}" mpfrPACKAGE="${1%%/}"
unset MPFR_ARCH_PATH
;; ;;
mpc-[0-9]* | mpc-system) mpc-[0-9]* | mpc-system)
mpcPACKAGE="${1%%/}" mpcPACKAGE="${1%%/}"
...@@ -169,7 +173,7 @@ addLib() ...@@ -169,7 +173,7 @@ addLib()
# Build GMP # Build GMP
# ================ # ================
echo "---------------" echo "---------------"
if [ -d $GMP_ARCH_PATH ] if [ -d "$GMP_ARCH_PATH" ]
then then
echo "Already built: $gmpPACKAGE" echo "Already built: $gmpPACKAGE"
elif _foamIsSystem $GMP_ARCH_PATH elif _foamIsSystem $GMP_ARCH_PATH
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment