Import/Export Jobs¶

Definition and Main Properties¶

An import job is a consistent set of instructions to the Import/Export engine, with a purpose of modifying data contents in the target by means of the source data, assignment expressions and underlying data processing. An Import/Export job consists of the general job definition and of at least one section containing assignment(s) for at least one target attribute.

An Import/Export job can be saved in an *.xml file using the ‘Save’ button for later reuse. It can also be opened from an *.xml file using the 'Load' button.

Figure: Creating a new job

To create a new job, click on "Add job" and define where the data will be imported from, location of this file, where the data will be imported to and customise details.

Job Properties¶

The following is a detailed description of the job definition parameters.

Job Name¶

When created, an import job gets a generic ID (name) Jobn, where "n" is a simple counter. By double-clicking the job ID turns editable, i.e. user-specified ID may be typed.

Job On/Off Toggle¶

A checkbox next to Job ID controls if the actual job will be included in the "Run" or not.

Source Type¶

This is a choice among available type of source files:

Shape file¶

The source can be a Shape file containing any feature type: point (containing data on e.g. network nodes), polyline (containing data on e.g. network links) or polygon (containing data on e.g. catchments).

As shape files contain one data table only, an import job with a Shape file as a source cannot include import of multiple tables.

Excel file¶

An EXCEL file may contain several data tables to be included in the job definition. Each column in a table must contain a text header in Row1 and the data in consecutive rows. All data in one column must be of the same type.

CAD file¶

AutoCAD proprietary format "DWG" is supported, as well as “DXF” format. Because it is not an open standard format, MIKE+ import supports it with some limitations. E.g. geometry and some simple attributes are supported, while labels are not supported.

Geodatabase¶

Import can access tables in the ESRI geodatabase

ODBC¶

Open Database Connectivity (ODBC) interface is a C programming language interface that makes it possible for applications to access data from a variety of database management systems (DBMSs). ODBC implementation in MIKE+ import supports the following sources:

MS Access
SQLite

Relevant drivers for reading these file formats are not installed with MIKE+. To import a Microsoft Access database, the user must therefore install the Microsoft Access Database Engine driver beforehand. To import a SQLite database, the user must install the SQLite ODBC driver.

The geometry for the features shown on the map must be defined in Well Known Text (WKT) format.

ISYBAU file¶

The ISYBAU exchange formats are used for the standardized exchange of data for the planning, construction and operation of wastewater facilities, originating from Germany. ISYBAU is a column-oriented exchange format in text form.

MIKE+ DB¶

MIKE+ database (SQLite or PostGIS) can be used as a source for data import, typically when the current project needs to be populated with data from another MIKE+ project.

Result Layer¶

This option is used for export of mapped 1D network results to external formats, e.g. to shape. 2D result layers cannot be exported from this tool (use instead the option to export to shape file, from the 'Layers and symbols' tree view).

SQL Server¶

This option is used to connect to a database hosted on a SQL Server.

The geometry for the features shown on the map must be defined in one of the following formats: SQL Geometry, Well Known Binary (WKB) format or Well Known Text (WKT) format.

Oracle spatial¶

This option is used to connect to an Oracle spatial database.

ICM (csv format)¶

This option is used to read and import individual .csv files previously exported from a ICM model setup (one file per ICM database table). The recommended workflow to import these data is as follows:

Export the ICM model setup to individual .csv files, all saved in a common folder
Create an import job in MIKE+, and select this folder as import source
Click the 'Automap sections' buttons: this will load a predefined configuration, to import the collection system data (rivers and 2D model data not yet configured)
Review the import configuration and adjust if necessary
Run the import.

Note

This import option allows importing network data to MIKE+, but differences of numerical methods between the two different products imply that some data may not be imported or may need further manual adjustments after the import, especially for example for Rainfall-Runoff modelling. Also note that no automatic conversion of units nor coordinate system is performed during this import.

Source¶

This is a path and filename pointing to the file containing the data to be imported. When clicking on "…" button, browser opens with a filter including only file types available for the currently selected source type. The following file types are accessible:

Source Type	File type(s)
Shape file	*.SHP
Excel file	*.XLSX
CAD file	.DWG, .DXF
ODBC	.MDB, .XLSX, *.SQLite
ISYBAU file	*.XML
MIKE+ DB	SQLite, PostGIS
ICM	*.csv
Result layer	-

Table: Import/Export job source types

When importing data from shape files or from ICM, the source can either be a specific file or a folder. When selecting a folder as a source, the exact file to import is selected in the definition of the section, and each section within the same job can use a different file.

When connecting to a SQL Server database, the '…' button opens a dialog to supply the server name, the user name and password for authentication. The name of the database must also be selected among the list of databases hosted on the server.

Figure: Connecting to a SQL Server database

When the SQL Server and MIKE+ are installed on the same machine and when the authentication for the SQL server uses 'Windows Authentication', the user name and password fields must remain empty.

When connecting to an Oracle Spatial database, the '…' button opens a dialog to supply the server name, the user name and password for authentication, as well as the service name. In order to connect to an Oracle Spatial database, the 64-bit version of "Oracle Data Provider for .NET" (ODP.Net) must be installed. It can be installed through ODAC installer or Oracle database client installer. So, when clicking the '…' button to connect to the server, you will first need to browse to the location of the installed file Oracle.DataAccess.dll.

Figure: Connecting to an Oracle spatial database

One import job can only point to one source file.

Source Mask¶

This allows for including only those source tables which table name contains the specified string. E.g. typing "msm*" (without quotes) will exclude all the tables from a source (e.g.MU Classic *.MDB file) except these containing the string "msm", i.e. collection model model tables. This is useful in cases when a source file contains many tables, where reducing the display to only relevant tables facilitates a better overview and a more efficient work.

Source text format¶

This option relates to converting input strings to numbers, e.g. when numeric data (e.g. pipe diameters) in some data sources are defined as "string" / text type, instead of "real number" (e.g. Excel sheet). In that case it is necessary to specify which character is used as decimal separator.

User may select among two possible options:

Use point decimal separator: this will always look for point "." decimal separator. Records containing text using comma "," as a separator will therefore be invalid and won't be imported)
Use computer’s separator: this will use the format of the decimal separator selected in Windows' region settings, on the local computer. With this option, the separator selected in the Windows settings must match the separator used in the source file to be imported.

Target Type¶

This provides similar choices as Source Type.

Target Mask¶

The same functionality as "Source Mask", applied on the target.

Use variables¶

User-specified variables are often useful in creating assignments. User can assign a certain value to a variable, in a same way as for any target attribute, i.e. by creating an expression containing applicable source attributes, constants, functions and operators. A variable can be used in assignments for other target attributes in the current section, or in any other section within the actual import job.

Variables may be of the following types:

String
Double
Int (i.e. integer)
Bool (i.e. Boolean)
DateTime

Figure: Example with two user-specified variables, "var1" and "var2"

Application of a variable in an assignment expression must be consistent with the variable type.

To use variables, change the option 'Use variables' to Yes, and add a variable to the import job using the button "Add variable". This creates a new line below "Use variables". A variable is defined by its name and type.

Note

Variables are not supported when exporting a result layer.

Correct topology¶

"Correct topology" refers to the processing of nodes and links data and creating a network. When activated, this option operates on the "Link" or Rivers table. For any other table, this option is of no relevance.

"Correct topology" works in two modes when creating the network connectivity:

Connection from coordinates
- In this mode links geometry (i.e. coordinates) is defined, but the network connectivity (i.e. connections to node IDs) is undefined. When ‘Correct topology’ is activated with this mode, the network connectivity is established by geographical proximity of nodes and links' ending points.
- If a node is within a specified snapping distance from a link's ending point, then a network connection for a link is established, either as "FromNode" (at the link's geometry start point) or "ToNode" (at the link's geometry end point). The established connection implies moving ("snapping") the links' end points to the nodes.
- This process will create a network which may include orphan nodes (i.e. nodes not connected to any link), orphan links (i.e. links without any of "FromNode" or "ToNode"), very short links, etc. Resolving such incompleteness and/or anomalies of the network is supported by a dedicated tool "Topology repair", after completed import process.
- For river networks, this option establishes the location of river structures on the river (i.e. assigns the River ID and chainage) according to the coordinates of the imported structures.
- Additional parameter to this mode is "Snapping distance". It defines a search radius for nodes and river structures to be "snapped". Snapping distance is defined in actual map units.
Coordinates from connection
- In this mode the network connectivity is defined by FROM and TO nodes, but the geometry (i.e. coordinates) of the links is not defined. In this case, the links geometry is defined as straight lines between FROM and TO nodes.
- For river networks, this option establishes the coordinates of the imported structures, according to the imported River ID and chainage.

Figure: Example with "topology" activated. The network will be created by connecting nodes with the link-ends founds within snapping distance.

Dissolved lines¶

This option is available when the source file is a shape file containing dissolved features, for example multiple polylines or multiple polygons defining the same item in the shape file. In this case, two options are offered:

Merge: with this option, the multiple objects defining a single item are merged. This option is especially relevant when a polyline has been digitized with multiple polylines describing the same feature, and in this case all the polylines defining a given feature / item are merged during the import. For example, if a river item is made of three dissolved polylines in the shape file, only one longer river will be created.
Import separately: with this option, the multiple objects defining a single item are split, i.e. one item is created for each object. For example, if a river item is made of three dissolved polylines in the shape file, three rivers will be created.