Open Source Contribution - Release 0.4 Part 1

Further Contributions to Pandas and pySearch

Throughout Release 0.4 I will continue to focus on contributing to Pandas and further expand the pySearch project. This release will be very similar to 0.3, however I will tackle different issues within these projects. One of the new things that I've never worked with and will be exploring, is CI (Continuous Integration).

Contributing to Pandas

Last week I started exploring linting errors within Pandas in further detail. When exploring these errors, I realized that Panda's contributors were causing some of these errors through attempts to better format their documentation, so it is easier to read. For example, one of the errors that came up dealt with "unnecessary indentation", although Panda's program that checks for these types of errors in docstrings identified the indentation as an error, the contributors put this indentation on purpose so the information is easier to read. Therefore, not all linting errors can be easily fixed and require discussion with Panda's moderators to find a better way to write the documentation without the need for such indentation, or keep ignoring some errors in favour of better readability. I will be further talking with moderators to see how we can tackle these issues.

One type of error that I fixed was, "Use only one blank line to separate sections or paragraphs". There were a total of 127 errors that dealt with this. In a massive project like Pandas, it is important to keep documentation and spacing consistent throughout the project. In addition, when the code consists of millions of lines of code, every line that can be removed results in better performance and saves space. My pull request consisted of fixing most of these errors. It was a rather tedious job because I had to go through many files and remove unnecessary blank lines within the docstrings. Sometimes finding these bugs was easy because I had a file and a line number as a reference to the general area of the docstring where the error occurs, however sometimes the lines were not shown or it was trickier to find certain docstrings as they are reused in different files but defined somewhere else.


In addition to the linting errors, I found an issue that got me very interested. The issue asked for the rest of the CircleCI checks to be merged into Azure Pipelines. After the rest of the checks are merged within Azure Pipelines, CircleCI will be removed from the Pandas project. I'm currently exploring different ways to achieve this and will talk in more detail about this issue in the next blog.

Plans for pySearch

The pySearch project has been refactored and restructured by the creator. We agreed that we will not merge any pull requests until this process was completed. So, now that he completed restructuring the project, I will need to go back and ensure that the changes in my pull requests will now reflect the new structure.

Furthermore, the experience that I will receive from my recent issue in Pandas will be helpful for the pySearch project as I will be able to help integrate Continuous Integration like Travis CI, CircleCI, and Azure Pipelines within the project and put some slack on the moderators. With CI, the moderators will no longer have to worry about new pull requests breaking the program.



Comments

Post a Comment

Popular posts from this blog

First Enhancement in Pandas

Image
First Enhancement in Pandas After spending the last few months fixing bugs in Pandas and writing tests, I decided that I wanted to try something a little bit different. I went on a hunt for enhancement issues to see if I can add another type of contribution to Pandas. A lot of enhancement requests that I found were hard and technical, so I stayed away from them for now. However, I found one that didn't require a whole new functionality to be added to the code, but asked to add an extra calculation in an existing function, and I proceeded to work on it. Issue https://github.com/pandas-dev/pandas/issues/21689 The issue deals with the pd.DataFrame.describe() function in Pandas. Currently, this function provides 8 summary statistics that are performed on the DataFrame that calls this function. These summary statistics include: count, mean, std, min, 25%, 50%, 75%, max. All of these summary statistics are useful in data analysis and provide a summary of the central t
Read more

Working with Incomplete MultiIndex keys in Pandas

Image
Incomplete MultiIndex keys in Dropna This week I produced a Pull Request for the bug in Pandas that I discussed last week. Although I think the solution that I provided can be further improved, it does fix the immediate issue of incomplete MultiIndex keys in the subset parameter of the "dropna()" function. I submitted a Pull Request to gather some feedback on my solution and improve it based on the moderator's discretion. Overview of the Issue https://github.com/pandas-dev/pandas/issues/17737 The creator of the issue brought to attention a bug that does not allow MultiIndex keys to be incomplete in the subset parameter of the "dropna()" function.  DataFrame.dropna(...) function removes missing values. If you pass in a "subset=..." parameter into it, it will remove rows based on the specified list of columns that are listed in the subset. I created some sample code, following the examples that the moderator gave and here a
Read more

Progress in Open Source

Image
Progress in Open Source In this blog I will talk about my journey and progress in open source development so far. I've been really enjoying working on real world projects that are used by many people every day. It is not only challenging as every Pull Request required me to perform research in order to understand the issue and solve it, but it is also rewarding because I acquire hands-on experience that is much needed and also contribute to the open source community that I am very fond of now. Open Source Timeline Summary I started exploring open source development during September of 2018. I had to learn a lot of useful tools that Git and Github offers, which now come in handy almost every day. I now know how to create meaningful issues that follow the specifications of the project, as well as create and structure my Pull Requests to the standards of the project that I'm working on. Furthermore learning how to properly manipulate version history using
Read more