README.md 1.94 KB
Newer Older
CHARLES's avatar
CHARLES committed
1
2


3
# eo-dataflow-manager
CHARLES's avatar
CHARLES committed
4
5

Checks online services for changed in data files, and downloads them when needed.
CHARLES's avatar
CHARLES committed
6
7
8

# Installation

CHARLES's avatar
CHARLES committed
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
## PIP configuration

This projects use dependencies that are not publicly available on pypi.org. Create / modify the following file to configure pip to fetch packages on `nexus-test.ifremer.fr`.

- Linux: `~/.config/pip/pip.conf`
- Windows: `%APPDATA%\pip\pip.ini`

```ini
[global]
index = https://nexus-test.ifremer.fr/repository/pypi-public-release/pypi
index-url = https://nexus-test.ifremer.fr/repository/pypi-public-release/simple

extra-index-url=
    https://nexus-test.ifremer.fr/repository/pypi-public-snapshot/simple
    https://nexus-test.ifremer.fr/repository/pypi-private-release/simple
    https://nexus-test.ifremer.fr/repository/pypi-private-snapshot/simple
```

CHARLES's avatar
CHARLES committed
27
28
29
30
31
32
33
## Conda

```
# Create conda env
conda env create -f environment.yaml

# Activate created env
34
conda activate eo-dataflow-manager
CHARLES's avatar
CHARLES committed
35
36
37

# Install the downloader
./setup.py install
38
39
40
41
42
43
44

```
## PostgreSQL
```
sudo -u postgres createuser --interactive downloader_daemon
sudo -u postgres createdb -U downloader_daemon downloader_db
sudo -u postgres createdb -U downloader_daemon downloader_bfi
CHARLES's avatar
CHARLES committed
45
```
CHARLES's avatar
CHARLES committed
46

CHARLES's avatar
CHARLES committed
47
48
49
50
### Conda link issues

If `./setup.py install` fails with a linker error related to _unlzw_, delete the file `$CONDA_PREFIX/compiler_compat/ld`

CHARLES's avatar
CHARLES committed
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

# Development

## Tooling

```
# Install developer tools
pip install -r requirements-dev.txt
```

### Linter

Using command-line:
```bash
# Run linter against current codebase
CHARLES's avatar
CHARLES committed
66
pylint -j8 --errors-only --output-format=colorized lib/
CHARLES's avatar
CHARLES committed
67
# The --errors-only flag only shows the errors
CHARLES's avatar
CHARLES committed
68
69

# Detect more issues:
CHARLES's avatar
CHARLES committed
70
pylint -j8 lib/
CHARLES's avatar
CHARLES committed
71
72
73
74
75
76
77
78
79
80
81
82
```

There are also plugins for many editors and IDEs.

### Code formatting

The code has been automatically formatted to follow PEP8 standard.
```bash
# Format every python source file in lib/ folder
autopep8 --in-place --recursive -j8 lib/
```

CHARLES's avatar
CHARLES committed
83
84
85
86
87
88
There are also plugins for many editors and IDEs.

### Unit testing

```
pytest lib/
89
```