Recently, I embarked on a learning journey, delving into Data Analysis and Data Science. Using tools such as Python, Numpy, Pandas, Matplotlib, and Seaborn, I decided to solidify my knowledge by undertaking the Data Analysis course at freeCodeCamp, aiming to achieve my first certification.
Simultaneously, I introduced myself to the universe of Elixir Nx. In this new journey, I explored tools like Nx, Explorer, and VegaLite, promising to add a unique dimension to my data analyses.
I will share a detailed tutorial in an upcoming post, providing practical insight into their application.
After completing the certification using Python, the idea of embracing a new challenge, I decided to tackle data analysis problems using the Elixir toolset exclusively.
My first project in this new phase is the Mean-Variance-Standard Deviation Calculator. An exciting opportunity to apply newly acquired concepts and explore the effectiveness of Elixir in handling complex statistical analysis challenges.
By sharing this experience, I hope to document my progress and encourage other enthusiasts to explore multiple languages and tools on their data analysis journey. I am eager to share the results and lessons learned throughout this exciting transition from Python to Elixir. Stay tuned for more updates and practical tutorials!
Setup
Mix.install([ {:nx, "~> 0.6.4"} ]) Challenge
Mean-Variance-Standard Deviation Calculator.
Create a function named calculate() in the MeanVarStd module that uses Elixir Nx to output the mean, variance, standard deviation, max, min, and sum of the rows, columns, and elements in a 3 x 3 matrix.
The input of the function should be a list containing 9 digits. The function should convert the list into a 3 x 3 numerical array, and then return a map containing the mean, variance, standard deviation, max, min, and sum along both axes and for the flattened matrix.
The returned dictionary should follow this format:
%{ mean: [axis1, axis2, flattened], variance: [axis1, axis2, flattened], standard_deviation: [axis1, axis2, flattened], max: [axis1, axis2, flattened], min: [axis1, axis2, flattened], sum: [axis1, axis2, flattened], } If a list containing less than 9 elements is passed into the function, it should raise a ValueError exception with the message: "List must contain nine numbers." The values in the returned map should be lists and not numerical arrays.
For example, calculate([0,1,2,3,4,5,6,7,8]) should return:
%{ mean: [[3.0, 4.0, 5.0], [1.0, 4.0, 7.0], 4.0], variance: [ [6.0, 6.0, 6.0], [0.6666666666666666, 0.6666666666666666, 0.6666666666666666], 6.666666666666667 ], standard_deviation: [ [2.449489742783178, 2.449489742783178, 2.449489742783178], [0.816496580927726, 0.816496580927726, 0.816496580927726], 2.581988897471611 ], max: [[6, 7, 8], [2, 5, 8], 8], min: [[0, 1, 2], [0, 3, 6], 0], sum: [[9, 12, 15], [3, 12, 21], 36] } Solution
Create a custom exception:
defmodule ValueError do defexception message: "bad argument in arithmetic expression" end The MeanVarStr module:
defmodule MeanVarStd do def calculate(list) do if length(list) == 9 do tensor = list |> Nx.tensor() |> Nx.reshape({3, 3}) %{ mean: mean(tensor), variance: variance(tensor), standard_deviation: standard_deviation(tensor), max: max(tensor), min: min(tensor), sum: sum(tensor) } else raise ValueError end end defp mean(tensor) do [ tensor |> Nx.mean(axes: [0]) |> Nx.to_list(), tensor |> Nx.mean(axes: [1]) |> Nx.to_list(), tensor |> Nx.mean() |> Nx.to_number() ] end defp variance(tensor) do [ tensor |> Nx.variance(axes: [0]) |> Nx.to_list(), tensor |> Nx.variance(axes: [1]) |> Nx.to_list(), tensor |> Nx.variance() |> Nx.to_number() ] end defp standard_deviation(tensor) do [ tensor |> Nx.standard_deviation(axes: [0]) |> Nx.to_list(), tensor |> Nx.standard_deviation(axes: [1]) |> Nx.to_list(), tensor |> Nx.standard_deviation() |> Nx.to_number() ] end defp max(tensor) do [ tensor |> Nx.reduce_max(axes: [0]) |> Nx.to_list(), tensor |> Nx.reduce_max(axes: [1]) |> Nx.to_list(), tensor |> Nx.reduce_max() |> Nx.to_number() ] end defp min(tensor) do [ tensor |> Nx.reduce_min(axes: [0]) |> Nx.to_list(), tensor |> Nx.reduce_min(axes: [1]) |> Nx.to_list(), tensor |> Nx.reduce_min() |> Nx.to_number() ] end defp sum(tensor) do [ tensor |> Nx.sum(axes: [0]) |> Nx.to_list(), tensor |> Nx.sum(axes: [1]) |> Nx.to_list(), tensor |> Nx.sum() |> Nx.to_number() ] end end Test
ExUnit.start(auto_run: false) defmodule ExampleTest do use ExUnit.Case, async: false # @tag :skip test "Expected different output when calling 'calculate()' with '[2,6,2,8,4,0,1,5,7]'" do actual = MeanVarStd.calculate([2, 6, 2, 8, 4, 0, 1, 5, 7]) expected = %{ max: [[8, 6, 7], [6, 8, 7], 8], mean: [ [3.6666667461395264, 5.0, 3.0], [3.3333332538604736, 4.0, 4.333333492279053], 3.8888888359069824 ], min: [[1, 4, 0], [2, 0, 1], 0], standard_deviation: [ [ 3.0912060737609863, 0.8164966106414795, 2.943920373916626 ], [ 1.8856180906295776, 3.265986442565918, 2.494438409805298 ], 2.6434171199798584 ], sum: [[11, 15, 9], [10, 12, 13], 35], variance: [ [ 9.555554389953613, 0.6666666865348816, 8.666666984558105 ], [ 3.555555582046509, 10.666666984558105, 6.222222805023193 ], 6.987654209136963 ] } assert actual == expected end # @tag :skip test "Expected different output when calling 'calculate()' with '[9,1,5,3,3,3,2,9,0]'" do actual = MeanVarStd.calculate([9, 1, 5, 3, 3, 3, 2, 9, 0]) expected = %{ max: [[9, 9, 5], [9, 3, 9], 9], mean: [ [ 4.666666507720947, 4.333333492279053, 2.6666667461395264 ], [5.0, 3.0, 3.6666667461395264], 3.8888888359069824 ], min: [[2, 1, 0], [1, 3, 0], 0], standard_deviation: [ [ 3.0912060737609863, 3.399346351623535, 2.054804801940918 ], [3.265986442565918, 0.0, 3.858612298965454], 3.034777879714966 ], sum: [[14, 13, 8], [15, 9, 11], 35], variance: [ [ 9.55555534362793, 11.555556297302246, 4.222222328186035 ], [10.666666984558105, 0.0, 14.888888359069824], 9.20987606048584 ] } assert actual == expected end # @tag :skip test "List must contain nine numbers." do assert_raise ValueError, "bad argument in arithmetic expression", fn -> MeanVarStd.calculate([2, 6, 2, 8, 4, 0, 1]) end end end Execute Test
PS: Remember to remove the @tag :skip.
ExUnit.run() ... Finished in 0.00 seconds (0.00s async, 0.00s sync) 3 tests, 0 failures Randomized with seed 433319 You can check out my livebook solution.
Top comments (3)
A really interesting article. Thanks for sharing!
Thanks, @airled glad you like the post! <3
Is there no easy way of achieving the same using Explorer?