Plsek commited on
Commit
15974f3
·
1 Parent(s): 6c6f7cf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -83
README.md CHANGED
@@ -14,80 +14,12 @@ The architecture of the convolutional network consists of 5 convolutional blocks
14
 
15
  ![Architecture](figures/architecture.png)
16
 
17
- ## Requirements
18
-
19
- For simple usage of the ***CADET*** pipeline, following libraries are required:\
20
- `matplotlib`\
21
- `astropy`\
22
- `numpy`\
23
- `scipy`\
24
- `sklearn`\
25
- `keras`\
26
- `tensorflow`
27
-
28
- If you want to re-train the network from scratch or generate training images, an additional library is required:\
29
- [`jax`](https://github.com/google/jax)
30
-
31
- ## Usage
32
-
33
- The ***CADET*** pipeline inputs either raw *Chandra* images in units of counts (numbers of captured photons) or exposure-corrected images. When using exposure-corrected images, images should be normalized by the lowest pixel value so all pixels are higher than or equal to 1. For images with many point sources, they should be filled with surrounding background level using Poisson statistics ([dmfilth](https://cxc.cfa.harvard.edu/ciao/ahelp/dmfilth.html) within [CIAO](https://cxc.harvard.edu/ciao/)).
34
-
35
- Convolutional part of the ***CADET*** pipeline can only input 128x128 images. As a part of the pipeline, input images are therefore being cropped to a size specified by parameter scale (size = scale * 128 pixels) and re-binned to 128x128 images. By default, images are probed on 4 different scales (1,2,3,4). The size of the image inputted into the pipeline therefore needs to at least 512x512 pixels (minimal input size differs if non-default scales are used) and images should be centred at the centre of the galaxy. The re-binning is performed using *Astropy* and *Numpy* libraries and can only handle integer binsizes. For floating point number binning, we recommend using [dmregrid](https://cxc.cfa.harvard.edu/ciao/ahelp/dmregrid.html) and applying ***CADET*** model manually (see Convolutional part).
36
-
37
- Before being decomposed by the DBSCAN algorithm, pixel-wise predictions produced by the convolutional part of the ***CADET*** pipeline need to be further thresholded. In order to simultaneously calibrate the volume error and false positive rate, we introduced two discrimination thresholds (for more info see [Plšek et al. 2023]()) and their default values are 0.4 and 0.6, respectively. Nevertheless, both discrimination thresholds are changeable and can be set to an arbitrary value between 0 and 1.
38
-
39
- The ***CADET*** pipeline is composed as a self-standing Python script (`CADET.py`), which can be run by simply calling it from a terminal using following arguments:\
40
- `filename` - string, name of the fits file\
41
- `scales` - list, list of size scales used to crop input images, optional (default: [1,2,3,4])\
42
- `threshold1` - float, between 0 and 1, calibrates volume error, optional (default: 0.4)\
43
- `threshold2` - float, between 0 and 1, calibrates false positive rate, optional (default: 0.6)
44
-
45
- ```console
46
- $ python3 CADET.py filename [scales] [threshold1] [threshold2]
47
- ```
48
-
49
- Example:
50
-
51
- ```console
52
- $ python3 CADET.py NGC5813.fits
53
- $ python3 CADET.py NGC5813.fits [1,2,3,4]
54
- $ python3 CADET.py NGC5813.fits [1,2,3,4] 0.5 0.9
55
- ```
56
-
57
- The `CADET.py` script loads a FITS file specified by the `filename` argument, which is located in the same folder as the main `CADET.py` script. The script creates a folder of the same name as the FITS file, and saves corresponding pixel-wise as well as decomposed cavity predictions into the FITS format while also properly preserving the WCS coordinates. On the output, there is also a PNG file showing decomposed predictions for individual scales.
58
-
59
- The volumes of X-ray cavities are calculated under the assumption of rotational symmetry along the direction from the galactic centre towards the centre of the cavity (estimated as *center of mass*). The cavity depth in each point along that direction is then assumed to be equal to its width. Thereby produced 3D cavity models are stored in the `.npy` format and can be used for further caclulation (e.g. cavity energy estimationš)
60
-
61
- ![](figures/NGC5813.png)
62
-
63
- ### Convolutional part
64
-
65
- <!-- [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tomasplsek/CADET/blob/main/CADET_example_colab.ipynb) -->
66
-
67
- The convolutional part of the pipeline can be used separately to produce raw pixel-wise predictions. Since the convolutional network was implemented using the functional *Keras* API, the architecture together with trained weights could have been stored in the HDF5 format (*CADET.hdf5*). Trained model can be therefore simply loaded using the `load_model` *Keras* function:
68
 
69
  ```python
70
- from keras.models import load_model
71
 
72
- model = load_model("CADET.hdf5")
73
-
74
- y_pred = model.predict(X)
75
- ```
76
-
77
- The CNN network inputs 128x128 images, however, to maintain the compatibility with *Keras*, the input needs to be reshaped as `X.reshape(1, 128, 128, 1)` for single image or as `X.reshape(-1, 128, 128, 1)` for multiple images.
78
-
79
- Thus produced pixel-wise prediction needs to be further thresholded and decomposed into individual cavities using a DBSCAN clustering algorithm:
80
-
81
- ```python
82
- import numpy as np
83
- from sklearn.cluster import DBSCAN
84
-
85
- y_pred = np.where(y_pred > threshold, 1, 0)
86
-
87
- x, y = y_pred.nonzero()
88
- data = np.array([x,y]).reshape(2, -1)
89
-
90
- clusters = DBSCAN(eps=1.5, min_samples=3).fit(data.T).labels_
91
  ```
92
 
93
  ## How to cite
@@ -103,15 +35,4 @@ The ***CADET*** pipeline was originally developed as a part of my [diploma thesi
103
  archivePrefix={arXiv},
104
  primaryClass={astro-ph.HE}
105
  }
106
- ```
107
-
108
- ## Todo
109
-
110
- The following improvements for the data generation and training process are currently planned:
111
-
112
- - [ ] add other features (cold fronts, complex sloshing, point sources, jets)
113
- - [ ] use more complex cavity shapes (e.g. [Guo et al. 2015](https://arxiv.org/abs/1408.5018))
114
- - [ ] train on multiband images simulated using PyXsim/SOXS
115
- - [ ] replace DBSCAN by using instance segmentation
116
- - [ ] restrict the cavity number and shape using regularization?
117
- - [ ] systematic cavity size uncertainty estimation using MC Dropout
 
14
 
15
  ![Architecture](figures/architecture.png)
16
 
17
+ ## How to use
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
 
19
  ```python
20
+ from huggingface_hub import from_pretrained_keras
21
 
22
+ model = from_pretrained_keras("Plsek/CADET-v1")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
  ```
24
 
25
  ## How to cite
 
35
  archivePrefix={arXiv},
36
  primaryClass={astro-ph.HE}
37
  }
38
+ ```