add option MESSAGE_CLASS, Sender and Receiver process compute message with type MESSAGE_CLASS #63

houzhizhen · 2021-06-20T04:48:01Z

add option MESSAGE_CLASS, Sender and Receiver process compute message with type MESSAGE_CLASS
delete ValueFactory, move the method in ValueFactory to GraphFactory

computer-core/src/main/java/com/baidu/hugegraph/computer/core/config/ComputerOptions.java

javeme · 2021-06-21T03:55:11Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/graph/BuiltinGraphFactory.java

+ return createValue(type);
+ }
+
+ // Can reuse Value


move to line 151, add TODO

update with "Create property value by type."

javeme · 2021-06-21T03:57:30Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/io/StreamGraphInput.java


- private Value<?> readValue(RandomAccessInput in) throws IOException {
+ private Value<?> readMessage(RandomAccessInput in) throws IOException {
+ Value<?> v = this.config.createObject(ComputerOptions.MESSAGE_CLASS);


also add createObject(String class) to GraphFactory? and then call graphFactory.createObject(ComputerOptions.MESSAGE_CLASS);

javeme · 2021-06-21T03:59:15Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/io/StreamGraphOutput.java

 value.write(out);
 }

+ private void writeMessage(RandomAccessOutput out, Value<?> value)


rename to writeValue

Both writeValue and writeMessage reserved, and it writes code in writeValue while does not in writeMessage.

javeme · 2021-06-21T04:06:55Z

computer-test/src/main/java/com/baidu/hugegraph/computer/core/receiver/ReceiverUtil.java

- Id id = graphInput.readMessage().getKey();
- input.seek(position);
- return id;
+ return StreamGraphInput.readId(input);


the StreamGraphInput.readId() is changed to static method?
the old code a id is prefix with bytes length, ensure it's expected now.

StreamGraphOutput doesn't write bytes length, only EntryOutput write length first.

codecov · 2021-06-21T09:41:26Z

Codecov Report

Merging #63 (62037f3) into master (ea4c743) will increase coverage by 0.08%.
The diff coverage is 92.42%.

@@ Coverage Diff @@ ## master #63 +/- ## ============================================ + Coverage 87.73% 87.81% +0.08%  + Complexity 2087 2083 -4  ============================================ Files 226 225 -1 Lines 7100 7083 -17 Branches 577 576 -1 ============================================ - Hits 6229 6220 -9  + Misses 573 566 -7  + Partials 298 297 -1

Impacted Files	Coverage Δ
...aidu/hugegraph/computer/core/config/HotConfig.java	`100.00% <ø> (+6.25%)`	⬆️
...h/computer/core/receiver/MessageRecvPartition.java	`81.42% <ø> (+2.26%)`	⬆️
...graph/computer/core/graph/BuiltinGraphFactory.java	`91.17% <82.35%> (-8.83%)`	⬇️
...egraph/computer/core/io/JsonStructGraphOutput.java	`93.97% <87.50%> (+0.22%)`	⬆️
...u/hugegraph/computer/core/io/StreamGraphInput.java	`97.40% <94.44%> (+5.51%)`	⬆️
...ugegraph/computer/core/common/ComputerContext.java	`100.00% <100.00%> (ø)`
...hugegraph/computer/core/graph/value/ListValue.java	`96.20% <100.00%> (-0.05%)`	⬇️
...hugegraph/computer/core/graph/value/ValueType.java	`91.30% <100.00%> (-0.37%)`	⬇️
...ph/computer/core/aggregator/DefaultAggregator.java	`82.85% <100.00%> (ø)`
...ugegraph/computer/core/config/ComputerOptions.java	`99.47% <100.00%> (-0.01%)`	⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea4c743...62037f3. Read the comment docs.

Linary · 2021-06-22T02:52:07Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/config/DefaultConfig.java

- @Override
- public ValueType valueType() {
- return this.hotConfig.valueType();
+ if (this.computation == null) {


can init this.computation at ctor?

javeme · 2021-06-22T05:45:06Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/io/JsonStructGraphOutput.java

 this.writeSplitter();

- String valueName = this.config.vertexValueName();
+ String valueName = this.config.get(ComputerOptions.OUTPUT_VALUE_NAME);


define as a field to avoid fetching from config every time

javeme · 2021-06-22T05:47:05Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/config/ComputerOptions.java

- "CUSTOM_VALUE.",
+ "algorithm.result_class",
+ "The class of vertex's value, the instance is used to " +
+ "store computation result for the vertex.",


computation result of each vertex

"computation result for the vertex" is more appropriate.

javeme · 2021-06-22T05:47:52Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/config/ComputerOptions.java


+ public static final ConfigOption<String> OUTPUT_VALUE_NAME =
+ new ConfigOption<>(
+ "output.value_name",


also rename to result_name?

javeme · 2021-06-22T05:48:58Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/graph/BuiltinGraphFactory.java

 case LIST_VALUE:
 return new ListValue<>();
- case CUSTOM_VALUE:
- return this.config.createObject(ComputerOptions.VALUE_CLASS);


prefer to keep the feature

No need, all type of property are builtin.

javeme · 2021-06-22T05:51:06Z

computer-core/src/main/java/com/baidu/hugegraph/computer/core/sender/QueuedMessageSender.java

 int busyClientCount = 0;
 for (WorkerChannel channel : channels) {
+ // The channel can't be null in online environment.
+ if (channel == null) {


when the channel is null?

In test case QueuedMessageSenderTest#testInitAndClose

javeme · 2021-06-22T05:53:09Z

computer-test/src/main/java/com/baidu/hugegraph/computer/core/sender/MultiQueueTest.java


 @Test
- public void testPutAndTakeWithHungry() throws InterruptedException {
+ public void testPutAndTakeWithPutAtFront() throws InterruptedException {


PutAtFront means what? is it testPutAtFront?

queue.putAtFront

javeme · 2021-06-22T06:02:00Z

computer-test/src/main/java/com/baidu/hugegraph/computer/core/sender/MultiQueueTest.java

 try {
- putLatch.await();
+ latches[0].countDown();
+ QueuedMessage m1 = queue.take();


improve var name

javeme · 2021-06-22T06:03:49Z

computer-test/src/main/java/com/baidu/hugegraph/computer/suite/unit/UnitTestSuite.java

+ public static void setup() throws ClassNotFoundException {
 LOG.info("Setup for UnitTestSuite of hugegraph-computer");
-
+ Class.forName(IdType.class.getName());


any better way to load it?

houzhizhen force-pushed the message branch from 8d25243 to 261b736 Compare June 21, 2021 01:41

javeme reviewed Jun 21, 2021

View reviewed changes

houzhizhen added 2 commits June 21, 2021 15:08

delete ValueFactory

79c6d92

rename VALUE_CLASS to ALGORITHM_RESULT_CLASS, remove VALUE_TYPE

3d51c00

houzhizhen force-pushed the message branch from 261b736 to 3d51c00 Compare June 21, 2021 09:36

houzhizhen added 2 commits June 21, 2021 20:29

bug fix

4444f58

improve coverage

533d4e6

Linary requested changes Jun 22, 2021

View reviewed changes

add option OUTPUT_VALUE_NAME

a56f0dd

javeme reviewed Jun 22, 2021

View reviewed changes

check channel not null in QueuedMessageSender#init

62037f3

javeme approved these changes Jun 22, 2021

View reviewed changes

Linary approved these changes Jun 22, 2021

View reviewed changes

Linary merged commit 429e081 into master Jun 22, 2021

Linary deleted the message branch June 22, 2021 08:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add option MESSAGE_CLASS, Sender and Receiver process compute message with type MESSAGE_CLASS #63

add option MESSAGE_CLASS, Sender and Receiver process compute message with type MESSAGE_CLASS #63

Uh oh!

houzhizhen commented Jun 20, 2021

Uh oh!

javeme Jun 21, 2021

houzhizhen Jun 21, 2021

javeme Jun 21, 2021

houzhizhen Jun 21, 2021

javeme Jun 21, 2021

houzhizhen Jun 21, 2021

javeme Jun 21, 2021

houzhizhen Jun 21, 2021

codecov bot commented Jun 21, 2021 •

edited

Loading

Linary Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

javeme Jun 22, 2021

houzhizhen Jun 22, 2021

Labels

3 participants

add option MESSAGE_CLASS, Sender and Receiver process compute message with type MESSAGE_CLASS #63

add option MESSAGE_CLASS, Sender and Receiver process compute message with type MESSAGE_CLASS #63

Uh oh!

Conversation

houzhizhen commented Jun 20, 2021

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Labels

3 participants

codecov bot commented Jun 21, 2021 •

edited

Loading