Skip to content

Duplicate path contexts in C# extractor #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jan 9, 2019
Merged

Duplicate path contexts in C# extractor #7

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ca576f6
Add sampling in GetInternalPaths
ASvyatkovskiy Jan 7, 2019
f3aebfa
Add uniqueness check in Variables
ASvyatkovskiy Jan 7, 2019
269de07
Use StreamWriter instead of standard out
ASvyatkovskiy Jan 7, 2019
39b7f1e
Add MaxContexts and output file parameters, add Reservoir sampling ut…
ASvyatkovskiy Jan 7, 2019
f5f3fba
Flush stream to write last line
ASvyatkovskiy Jan 8, 2019
ba7174a
Pass ofile_name command line option to the python script
ASvyatkovskiy Jan 9, 2019
a909b15
Change variable names to follow convention used
ASvyatkovskiy Jan 9, 2019
bd03dda
Specify ofile_name argument instead of stdout
ASvyatkovskiy Jan 9, 2019
f7a4689
Use a file pointed to by ofile_name rather than directing from stdout
ASvyatkovskiy Jan 9, 2019
ca6dcbf
Revert change in the Pool size
ASvyatkovskiy Jan 9, 2019
10ca08c
Use IDisposable to manage StreamWriter
ASvyatkovskiy Jan 9, 2019
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Specify ofile_name argument instead of stdout
  • Loading branch information
@ASvyatkovskiy
ASvyatkovskiy committed Jan 9, 2019
commit bd03dda94df1949323343cb7b57a0dfb6ffc6f0d
6 changes: 3 additions & 3 deletions preprocess_csharp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,13 @@ mkdir -p data
mkdir -p data/${DATASET_NAME}

echo "Extracting paths from validation set..."
${PYTHON} CSharpExtractor/extract.py --dir ${VAL_DIR} --max_path_length 8 --max_path_width 2 --num_threads ${NUM_THREADS} --csproj ${EXTRACTOR_JAR} > ${VAL_DATA_FILE}
${PYTHON} CSharpExtractor/extract.py --dir ${VAL_DIR} --max_path_length 8 --max_path_width 2 --num_threads ${NUM_THREADS} --csproj ${EXTRACTOR_JAR} --ofile_name ${VAL_DATA_FILE}
echo "Finished extracting paths from validation set"
echo "Extracting paths from test set..."
${PYTHON} CSharpExtractor/extract.py --dir ${TEST_DIR} --max_path_length 8 --max_path_width 2 --num_threads ${NUM_THREADS} --csproj ${EXTRACTOR_JAR} > ${TEST_DATA_FILE}
${PYTHON} CSharpExtractor/extract.py --dir ${TEST_DIR} --max_path_length 8 --max_path_width 2 --num_threads ${NUM_THREADS} --csproj ${EXTRACTOR_JAR} --ofile_name ${TEST_DATA_FILE}
echo "Finished extracting paths from test set"
echo "Extracting paths from training set..."
${PYTHON} CSharpExtractor/extract.py --dir ${TRAIN_DIR} --max_path_length 8 --max_path_width 2 --num_threads ${NUM_THREADS} --csproj ${EXTRACTOR_JAR} > ${TRAIN_DATA_FILE}
${PYTHON} CSharpExtractor/extract.py --dir ${TRAIN_DIR} --max_path_length 8 --max_path_width 2 --num_threads ${NUM_THREADS} --csproj ${EXTRACTOR_JAR} --ofile_name ${TRAIN_DATA_FILE}
echo "Finished extracting paths from training set"

TARGET_HISTOGRAM_FILE=data/${DATASET_NAME}/${DATASET_NAME}.histo.tgt.c2v
Expand Down