Skip to content
This repository was archived by the owner on May 25, 2023. It is now read-only.

Conversation

@debasishg
Copy link
Contributor

Some suggested by @deanwampler. Added documentation on how to run applications in distributed mode using the query layer.

Copy link

@deanwampler deanwampler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thanks for filling in the distributed section. I made some minor comments...


```bash
$ sbt
$ dslPackage/universal:packageZipTarball

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use > as the prompt here inside sbt, like you normally do, to avoid potential confusion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done ..


1. Zookeeper and Kafka are running
2. All topics mentioned above are created
3. The folder mentioned in `directoyToWatch` in `application.conf` has the data file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

directoryToWatch (missing second r)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done ..


This starts the single instance of the application. After some time you will see data printed in the console regarding the host access information as present from the data file.

In the log file, created under `<...>/dslpackage-0.0.1/logs`, check if the REST service has started and note the host and port details. It should be something like `localhost:7070` (the default setting in `application.conf`).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You suggest they edit the logback.xml file above. If the REST endpoint is written using a log statement, is it written with warning, or something like that, so the user is unlikely to suppress it accidentally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it's at the INFO level - will change to WARN - good suggestion ..


### Step 3: Run the second instance of the application

If you decide to run multiple instances of the application you may choose to split the dataset into 2 parts and keep them in different folders. Also you need to copy the current distribution in some other folder and start the seocnd instance from there, since you need to run it with changed settings in `application.conf`. Say we want to copy in a folder named `clarknet-2`.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

second (typo)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done ..


```bash
$ sbt
$ procPackage/universal:packageZipTarball

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

> for sbt prompt (as before...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done ..


1. Zookeeper and Kafka are running
2. All topics mentioned above are created
3. The folder mentioned in `directoyToWatch` in `application.conf` has the data file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

directoryToWatch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done ..


This starts the single instance of the application. After some time you will see data printed in the console regarding the host access information as present from the data file.

In the log file, created under `<...>/procpackage-0.0.1/logs`, check if the REST service has started and note the host and port details. It should be something like `localhost:7070` (the default setting in `application.conf`).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as previously about ensuring the REST message isn't suppressed if the log threshold is set high (like WARN).

Or, is it possible to write this URL to the console?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to WARN and also added a println which will print to the console


Here are a few examples:

```bash

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you put these bash comments after the commands in the other README. True? I actually prefer them before the commands, like you have here, but I'm just suggesting you do it consistently, one way or the other.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now all the comments are before the commands ..

@debasishg
Copy link
Contributor Author

Thanks .. I will work on these changes and then merge.

@debasishg debasishg merged commit 9a25cc0 into develop Dec 6, 2017
@debasishg debasishg deleted the refinements branch December 6, 2017 18:28
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

4 participants