Just KNIME It!
Prove your KNIME knowledge and practice your workflow building skills by solving our weekly challenges!
Leave your email to be notified when a new challenge comes out.
Sign me up
Challenge 07: Mapping Successful Football Passes
Level: Easy
Description: By visually analyzing successfully completed passes from football match data, it is possible to obtain valuable information about a team's performance. This can in turn be used by coaches and team managers to make informed decisions about how to improve their game. In this challenge, your task is to generate a visualization that adequately showcases the starting positions of successful passes made by the players of Manchester City team. To this end, you will use data that includes both starting (x and y) and ending (endX and endY) positions of various types of passes made during a football match between Manchester City and Manchester United.
Author: Keerthan Shetty
Dataset: Data on Football Passes in the KNIME Hub
Remember to upload your solution with tag JKISeason2-7 to your public space on the KNIME Community Hub. To increase the visibility of your solution, also post it to this challenge thread on the KNIME Forum.
We will post our solution to this challenge here next Tuesday.
Go to the KNIME Forum
Here is how the challenges work:
⇒ We post a challenge on Wednesday.
⇒ You create a solution with KNIME Analytics Platform.
⇒ Upload it to your public KNIME Community Hub Space.
⇒ Check your rank on the Just KNIME It Leaderboard.
Our solution to the challenge comes out on the following Tuesday.
Just KNIME It! Leaderboard
KNIME community members are working hard to solve the latest "Just KNIME It!" challenge - and some of you have solved dozens of them already! Who are the KNIME KNinjas who have completed the most challenges? Click over to the leaderboard on the KNIME Forum to find out! How many challenges have you solved?
Level: Easy
Description: Earth Day just passed and we need to raise awareness for air quality and water pollution. In this context, you decide to explore a 2020 dataset in which users voted about air quality and water pollution for a number of different cities. The scale for air quality goes from 0 (bad quality) to 100 (top quality), and the scale for water pollution also goes from 0 (no pollution) to 100 (extreme pollution). Your task is to rank and visualize the top 10 countries twice with different criteria: first, based on how bad the air quality is; and second, based on how polluted the water is.
Author: Daria Liakh
Dataset: Air Quality and Water Polution Data in the KNIME Hub
Solution Summary: To tackle this challenge, we group the data by country, getting the mean air quality and water pollution levels to represent each one of them. Next, we sort the resulting data in two different ways: first, by air quality in ascending order; second, by water pollution level in descending order. Finally, we select the top 10 countries with worst (mean) air quality and worst (mean) water pollution levels, and visualize the corresponding information with two bar plots. In our solution, Benin turned out to be one of the countries with highest water pollution levels, and Central African Republic is the country with worst air quality.
Solution Details: After reading the dataset with the CSV Reader node, we remove unnecessary region information with the Column Filter node and then group the city-level information by country with the GroupBy node, such that in the end we have the mean levels of air quality and water pollution for each one of the countries in the dataset. Next, we use two instances of the Top k Selector node to sort the data by air quality (ascending order) and by water pollution level (descending order). We send the resulting data to a component in order to get a composite view with two bar plots: one for the countries with worst air quality, and one for the countries with worst air pollution levels. We use two instances of the Bar Chart (Labs) node for this composite view.
Level: Easy
Description: Your team created a summary of the total amount spent last year on each project. The data looks like the following:
Year Area Project exp
2022 area A proj A 120
2022 area B proj X 160
The finance department of your company, however, would like to have a more granular overview of the expenditures. In particular, the expenditures for projects in area A should be spread equally across 12 months, while those for projects in area B should be grouped by quarters. In the end, the above table should look like the following:
Year Area Project exp
2022 area A proj A jan 10
2022 area A proj A feb 10
...
2022 area B proj X Q1 40
2022 area B proj X Q2 40
...
Dataset: Expenditures Data on KNIME Community Hub
Solution Summary: To solve this challenge, we started by separating the data into Area A and Area B. Next, we created auxiliary tables for the year’s months and quarters, and combined them with the 12th part of each project’s expenditures in the case of Area A, and the 4th part of each project’s expenditures in the case of Area B. Finally, we concatenated the combined information into a single table.
Solution Details: After reading the dataset for this challenge with the Table Reader node, we use the Row Splitter node to split it based on Area (Area A or Area B). We then forward the rows that correspond to Area A to the Math Formula node, which divides the expenditures of each project in Area A by 12. In parallel, we use the Table Creator node to create labels for the 12 months of the year, and then combine the expenditures’ fractions for the projects in Area A with these labels by using the Cross Joiner node. As a result, we end up with a table that has the expenditures for each project in Area A equally spread across 12 months. Similarly, we forward the rows for projects in Area B to another instance of the Math Formula node, which divides their corresponding expenditures by 4. Another instance of the Table Creator node creates labels for each quarter of the year, and the Cross Joiner node is again used to combine expenditures’ fractions and quarter labels. In the end, the expenditures for projects in Area B are equally spread across the quarters. These two more granular tables are then finally combined with the Concatenate node. Note: The Cross Joiner node is very expensive as the number of rows in the output is the product of both input table row counts. If executed in streaming mode, only the top input will be processed in a streamable fashion.
Level: Medium
Description: The sales manager at ABC Grocery Stores is looking for a way to compare the number of sales of a selected month with those of a prior month. You are tasked to create a component in KNIME which carries out this task, with a visualization tackling the comparisons. If there are no prior months then only data for the selected month should be shown. If a prior month can be picked for comparison, it should not be more than 12 months before the selected month.
Author: Ali Marvi
Dataset: Monthly Sales Data on KNIME Community Hub
Solution Summary: To solve this challenge, we built a component that allows users to (1) select the month against which they want to compare sales numbers, and (2) a second previous month based on a gap. As an example, users can pick June 2020 for (1) and, by entering a gap of -3, also pick March 2020 as (2) for comparisons. The component also contains a bar chart to compare sales numbers for both selected months, and a refresh button that allows users to vary their selections and resulting plots.
Solution Details: After the data is read with the Excel Reader node, it is fed to the component we built (Comparing Monthly Sales). Inside this component, we have three widget nodes to capture user’s inputs: the Date&Time Widget node for a month selection, the Integer Widget node for the gap selection, and the Refresh Button Widget node at the very beginning of the flow to allow users to vary their comparisons. Next, all inputs given by a user are merged with the Merge Variables node, and their values are captured with the Variable to Table Row node. These values are preprocessed with the String to Date&Time and Date&Time Shift nodes in order to properly filter the data based on the given input. After a bit more of preprocessing, the filtering is finally performed with the Reference Row Filter node, which takes the original data and the two selected months as input and outputs the corresponding data rows (months and sales numbers). The dates are converted back into strings with the Date&Time to String node to be compatible with what the Bar Chart node takes as input. This node finally plots the sales numbers for the two currently selected months.
Level: Medium
Description: Credit Card company ABC maintains information about customer purchases and payments. The information is available for individual customers as Payments Info and Purchase Info. The company wants to segment the customers into three (3) clusters, so that marketing campaigns can be designed according to each cluster. You are asked to use both infos together to build a clustering model that adequately segments the customers. What patterns do customers in the same cluster have in common? Also, Information for newly registered customers is available. You are asked to assign cluster labels to newly registered customers using the trained clustering model, and then export the results into a CSV file. Do the assignments make sense? How do you assess their quality?
Author: Mahantesh Pattadkal
Dataset: Customer Data on Kaggle
Solution Summary: To tackle this challenge, we start by partitioning the data into existing and new customers. Next, we preprocess the data for existing customers to address missing values and also to normalize it, facilitating its posterior clustering. We then use k-Means to identify three (k = 3) clusters in this data, whose quality we evaluate by calculating the Silhouette coefficient (the value was close enough to 1, so the clusters had acceptable quality). We also visualize the identified clusters with a scatter plot. Finally, we assign the identified cluster IDs to the new customers, grouping them as well.
The clustering was helpful to segment customers into categories low profile (Cluster_1, with users who have low order payments and few purchases); potential (Cluster_0, with users who have made medium to high order payments but a relatively low number of purchases); and premium (Cluster_2, with users who made high order payments and a high number of purchases). Marketing campaigns could focus on these different categories in order to boost revenue.
Solution Details: We start our solution by using the Partitioning node in order to separate the data into existing and new customers. Next, we use the Missing Value and Normalizer nodes in order to not only preprocess the data for existing customers, but also to create models to clean the new customer data (Missing Value (Apply) and Normalizer (Apply) nodes). After clustering the preprocessed data for existing customers with the k-Means node, we denormalize it (Denormalizer node) and visualize it (Color Manager and Scatter Plot nodes). Additionally, we also assess the quality of the clusters identified with k-Means with the Silhouette Coefficient node. Since the quality is good, we assign the identified cluster IDs to the preprocessed new customer data (Cluster Assigner node). We then denormalize this data (Denormalizer node) and write it to a CSV file with the assigned cluster IDs (CSV Writer node).
Level: Easy
Description: You are doing research on trading and different currencies and one of your tasks is to compare the daily exchange rates of US dollar vs 8 other currencies from 1980 to 1998. The data contains some missing values due to public holidays on weekdays, which you should handle first using a strategy of your choice. After that, you should calculate the correlations between the exchange rates of the currencies and visualize them, Which currencies had the highest positive/negative correlation? Which currencies were not correlated, or had a low correlation?
Author: Aline Bessa
Dataset: Exchange Rates Data on KNIME Community Hub
Solution Summary: After reading the dataset with daily exchange rates, we used the simple strategy of replacing missing values with previous dates'. Next, we computed the linear correlations across all currency pairs using Pearson's coefficient, and then visualized them with a heatmap. The highest positive correlations involved the Swiss, German, and Dutch currencies. The highest negative correlation was between the Australian and Japanese currencies. The correlations closest to zero were between the Canadian currency and the Dutch/German currencies.
Solution Details: We started to tackle this challenge by using the Table Reader node to read the dataset with daily exchange rates. Next, we used the Missing Value node to replace each missing value with its previous non-missing one, and then computed Pearson correlations across currency pairs using the Linear Correlation node. Finally, we used the Heatmap node to visualize these correlations in an interactive fashion.
Level: Hard
Description: You work for a Marketing agency that monitors the online presence of a few airline companies to understand how they are being reviewed. You were asked to identify whether a tweet mentioning an airline is positive, neutral, or negative, and decided to implement a simple sentiment analysis classifier for this task. What accuracy can you get when automating this process? Is the classifier likely to help company reviewers save their time? Note: Given the size of the dataset, training the classifier may take a little while to execute on your machine (especially if you use more sophisticated methods). Feel free to use only a part of the dataset in this challenge if you want to speed up your solution. Hint 1: Check our Textprocessing extension to learn more about how you can turn tweets' words into features that a classifier can explore. Hint 2: Study, use, and/or adapt shared components Enrichment and Preprocessing and Document Vectorization (in this order!) if you want to get a part of the work done more quickly. They were created especially for this challenge. Hint 3: Remember to partition the dataset into training and test set in order to create the decision tree model and then evaluate it. Feel free to use the partitioning strategy you prefer.
Author: Aline Bessa
Dataset: Airline Reviews Dataset in the KNIME Hub
Solution Summary: We tackle this challenge by preprocessing the tweets in order to turn them into term frequency bags of words, such that words in tweets can be explored as features by classifiers. We then partition the data into training and testing and use an AutoML component to get the best model (and parameters) for the given training data. In our case, this model was a Logistic Regressor. After applying this model over the testing data, we got an accuracy of 77%.
Solution Details: After reading the tweets from airline consumers (Table Reader node), we enrich the text with tags for positive and negative words, remove punctuation and stop words, perform stemming, and execute other text preprocessing tasks (Enrichment and Preprocessing component). Document vectors are then created to encode each tweet into a vector space. Next, the processed tweets are turned into term frequency bags of words, and each one of them is associated with its sentiment label (class) (Document Vectorization component). We then partition the data using stratified sampling on sentiment (class), keeping 80% for training and 20% for testing (Partitioning node). The training data is sent to an AutoML component that compares different classifiers and different parametrization/processing options. In the end, for that training data, the best model was a Logistic Regressor. We apply the model over the testing data (Workflow Executor node) and get an accuracy of 77% (Scorer node).
