@@ -13,28 +13,159 @@ Getting Started
1313Installation
1414-----------------------------------------------
1515
16- You can install LogAI using :file: `pip install ` with the instruction below :
16+ You can install LogAI core library using :file: `pip install `:
1717
1818.. code-block :: shell
1919
20- git clone https://git.soma.salesforce.com/SalesforceResearch/logai.git
21- cd logai
22- python3 -m venv venv # create virtual environment
23- source venv/bin/activate # activate virtual env
24- pip install ./ # install LogAI from root directory
20+ # Check out LogAI code repo from Github
21+ git clone https://git.soma.salesforce.com/SalesforceResearch/logai.git
22+ cd logai
2523
26-
24+ # [Optional] Create virtual environment
25+ python3 -m venv venv
26+ source venv/bin/activate
27+
28+ # Install LogAI
29+ pip install logai
30+
31+
32+ -----------------------------------------------
33+
34+ LogAI core library is light-weight with limited dependent packages installed. Users can install optional dependencies
35+ to enable extended functionalities of LogAI.
36+
37+ **Deep Learning Log Analysis **. To conduct deep learning model related tasks and run benchmarking,
38+ please install extra requirements by :file: `pip install "logai[deep-learning]" `.
39+
40+ **Enable LogAI GUI portal* **. To use LogAI GUI portal,
41+ please install extra requirements by :file: `pip install "logai[gui]" `.
42+
43+ **LogAI Development **. To contribute to LogAI development, build and test code changes,
44+ please install extra requirements by :file: `pip install "logai[dev]" `.
45+
46+ **Complete installation **. you can install the full list of dependencies by :file: `pip install "logai[all]" `.
47+
48+ Use LogAI
2749-----------------------------------------------
2850
51+ Below we briefly introduce several ways to explore and use LogAI, including exploring LogAI GUI
52+ portal, benchmarking deep-learning based log anomaly detection using LogAI, and building your
53+ own log analysis application with LogAI.
54+
55+
56+ Explore LogAI GUI Portal
57+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
58+
2959You can also start a local LogAI service and use the GUI portal to explore LogAI.
3060
3161.. code-block :: shell
3262
33- export PYTHONPATH=' .' # make sure to add current root to PYTHONPATH
34- python3 gui/application.py # Run local plotly dash server.
63+ # Check out LogAI code repo from Github
64+ git clone https://git.soma.salesforce.com/SalesforceResearch/logai.git
65+ cd logai
66+
67+ # [Optional] Create virtual environment
68+ python3 -m venv venv # create virtual environment
69+ source venv/bin/activate # activate virtual env
70+
71+ # install LogAI and GUI dependencies
72+ pip install " .[dev]"
73+ pip install " .[gui]"
74+
75+ # Start LogAI service
76+ export PYTHONPATH=' .' # make sure to add current root to PYTHONPATH
77+ python3 gui/application.py # Run local plotly dash server.
78+
3579
3680:file: `http://localhost:8050/ ` or :file: `http://127.0.0.1:8050/ ` in your browser:
3781
3882.. image :: _static/logai_summarization_res.png
3983 :width: 750
4084
85+ Run Simple Time-series Anomaly Detection Application
86+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
87+
88+ You can also use LogAI in more programtic ways. LogAI supports configuration files in `.json ` or `.yaml `.
89+ Below is a sample `log_anomaly_detection_config.json ` configuration for anomaly detection application.
90+ Make sure to set `filepath ` to the target log dataset file path.
91+
92+ .. code-block :: json
93+
94+ {
95+ "open_set_data_loader_config" : {
96+ "dataset_name" : " HDFS"
97+ "filepath" : " "
98+ },
99+ "preprocessor_config" : {
100+ "custom_delimiters_regex" :[]
101+ },
102+ "log_parser_config" : {
103+ "parsing_algorithm" : " drain"
104+ "parsing_algo_params" : {
105+ "sim_th" : 0.5 ,
106+ "depth" : 5
107+ }
108+ },
109+ "feature_extractor_config" : {
110+ "group_by_category" : [" Level"
111+ "group_by_time" : " 1s"
112+ },
113+ "log_vectorizer_config" : {
114+ "algo_name" : " word2vec"
115+ },
116+ "categorical_encoder_config" : {
117+ "name" : " label_encoder"
118+ },
119+ "anomaly_detection_config" : {
120+ "algo_name" : " one_class_svm"
121+ }
122+ }
123+
124+
125+
126+
127+
128+ .. code-block :: python
129+
130+ import json
131+
132+ from logai.applications.application_interfaces import WorkFlowConfig
133+ from logai.applications.log_anomaly_detection import LogAnomalyDetection
134+
135+ # path to json configuration file
136+ json_config = " ./log_anomaly_detection_config.json"
137+
138+ # Create log anomaly detection application workflow configuration
139+ config = json.loads(json_config)
140+ workflow_config = WorkFlowConfig.from_dict(config)
141+
142+ # Create LogAnomalyDetection Application for given workflow_config
143+ app = LogAnomalyDetection(workflow_config)
144+
145+ # Execute App
146+ app.execute()
147+
148+
149+ :file: `app.anomaly_results `.
150+
151+ For full context of this example please check
152+ `Tutorial: Use Log Anomaly Detection Application
153+ <https://github.com/salesforce/logai/blob/main/examples/jupyter_notebook/log_anomaly_detection_example.ipynb> `_.
154+
155+ Build Customized LogAI Applications
156+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
157+ You can build your own customized log analysis applications using LogAI. Here we show two examples:
158+
159+ * `Tutorial: Log Clustering Using LogAI <https://github.com/salesforce/logai/blob/main/examples/jupyter_notebook/tutorial_log_clustering.ipynb >`_
160+
161+ * `Tutorial: Log Anomaly Detection Using LogAI <https://github.com/salesforce/logai/blob/main/examples/jupyter_notebook/tutorial_log_anomaly_detection.ipynb >`_
162+
163+ Deep-learning Anomaly Detection Benchmarking
164+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
165+
166+ LogAI can be used to benchmark deep-learning anomaly detection results.
167+ A `tutorial <https://github.com/salesforce/logai/blob/main/examples/jupyter_notebook/tutorial_deep_ad.md >`_ is provided for
168+ Anomaly Detection Benchmarking using LSTM anomaly detector for HDFS Dataset. More examples of deep-learning anomaly
169+ detection benchmarking on different datasets and algorithms can be found in
170+ `Deep Anomaly Detection Benchmarking Examples <https://github.com/salesforce/logai/tree/main/examples/jupyter_notebook/nn_ad_benchmarking >`_.
171+
0 commit comments