+++
title = "Data import"
weight = 10
+++

## Overview

BornAgain provides functions to read 1D and 2D data fields:

```python
ba.readData1D(fname)
ba.readData1D(fname, ftype)
ba.readData1D(fname, ftype, import_settings)

ba.readData2D(fname)
ba.readData2D(fname, ftype)
ba.readData2D(fname, ftype, import_settings)
```

The single-argument forms determine the file format from the file name
extension (see tables below).
If the name ends with `.gz` or `.bz2`, the file is decompressed on the fly.

The multi-argument forms accept an explicit format constant (`ftype`),
needed when the file extension does not match the format or is not specific enough.
A third, optional argument is an `ImportSettings` object
that specifies in particular the column layout.
It is always required for 1D CSV files.

## Supported formats

##### 1D formats (for specular reflectometry)

| Format | Extension | Enum constant |
|--------|-----------|---------------|
| BornAgain internal text format | `.int` | `ba.bornagain1D` |
| Motofit format | `.mft` | `ba.mft` |
| CSV/ASCII column data | any other | `ba.csv1D` |

##### 2D formats (for GISAS/diffraction)

| Format | Extension | Enum constant |
|--------|-----------|---------------|
| BornAgain internal text format | `.int` | `ba.bornagain2D` |
| [TIFF image](tiff) | `.tif`, `.tiff` | `ba.tiff` |
| BERsans format | `.001` | `ba.nicos2D` |
| CSV/ASCII matrix data | any other | `ba.csv2D` |

## Reading 1D data

For CSV/ASCII column files, `ImportSettings1D` is always required
(the single-argument form throws for CSV, as it cannot infer column layout).
Pass format and settings explicitly:

```python
flags = ba.ImportSettings1D("2alpha (deg)", "#", "", 1, 2)
data = ba.readData1D(fname, ba.csv1D, flags)
```

For self-describing formats (`.int`, `.mft`), no settings are needed:

```python
data = ba.readData1D("reflectivity.mft")
```

**Examples:**
[Specular1Par.py][spec1par],
[Honeycomb_fit.py][honeycomb],
[Pt_layer_fit.py][ptlayer],
[TREFF_Ni_film.py][treff].

##### ImportSettings1D

`ImportSettings1D` describes the column layout of a CSV reflectivity file.
Column indices count from 1.

```python
flags = ba.ImportSettings1D(
    xCoord,       # coordinate type, e.g. "q_z (1/nm)" or "2alpha (deg)"
    headerPrefix, # prefix that marks header lines, e.g. "#"
    linesToSkip,  # line-number pattern to skip, e.g. "1,10-12"
    col_Q,        # column number of Q (or angle)
    col_R,        # column number of R (intensity)
    col_sR=0,     # column of sigma_R (uncertainty), 0 = absent
    col_dQ=0,     # column of delta_Q (resolution), 0 = absent
    col_lambda=0, # column of wavelength, 0 = absent
    sort=True,    # sort rows by Q
    rm_negative=True,    # discard rows with negative Q
    rm_duplications=True # discard rows with duplicate Q
)
```

## Reading 2D data

The simplest call uses automatic format detection from the file name:

```python
data = ba.readData2D("detector_image.tif.gz")
```

An explicit file type can be given as second argument.
For CSV format, `ImportSettings2D` is also required:

```python
data = ba.readData2D(fname, ba.csv2D, import_settings)
```

**Examples:**
[expfit_galaxi.py][galaxi] loads a TIFF file and fits a GISAS pattern
from the GALAXI instrument.
[fit_gisas.py][fitgisas] loads a BornAgain internal-format file.

##### ImportSettings2D

When reading a 2D CSV file, `ImportSettings2D` controls how axes are parsed:

```python
settings = ba.ImportSettings2D(
    xCoord,           # coordinate type of the x axis, e.g. "phi_f (deg)"
    yCoord,           # coordinate type of the y axis, e.g. "alpha_f (deg)"
    has_axes=False,   # True if the file contains explicit axis values
    swap_axes=False,  # True to swap x and y
    first_row=True,   # True if axis values are in the first row (else last)
    first_col=True,   # True if axis values are in the first column (else last)
)
```

## Reading using fabio and NumPy

If your instrument produces a format not listed above, read the data
into a numpy array with a third-party library and wrap it in a
`Datafield` before passing it to `FitObjective`:

```python
import fabio  # example: EDF/CBF/HDF5 and many more
import bornagain as ba

img = fabio.open("experimental_data.edf")
arr = img.data.astype("float64")
nrows, ncols = arr.shape
frame = ba.Frame(ba.EquiDivision("x", ncols, 0.0, float(ncols)),
                 ba.EquiDivision("y", nrows, 0.0, float(nrows)))
data = ba.Datafield(frame, list(arr.flatten()))
```

Then pass the `Datafield` to the fit engine:

```python
fit_objective = ba.FitObjective()
fit_objective.addFitPair(simulation_builder, data)
```

The array must match the detector pixel layout defined in the
simulation: same number of rows and columns.

[galaxi]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/expfit_galaxi.py
[fitgisas]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/fit_gisas.py
[spec1par]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Specular1Par.py
[honeycomb]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Honeycomb_fit.py
[ptlayer]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Pt_layer_fit.py
[treff]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/TREFF_Ni_film.py
