Open Source Contribution - Release 0.4 Part 1

Further Contributions to Pandas and pySearch

Throughout Release 0.4 I will continue to focus on contributing to Pandas and further expand the pySearch project. This release will be very similar to 0.3, however I will tackle different issues within these projects. One of the new things that I've never worked with and will be exploring, is CI (Continuous Integration).

Contributing to Pandas

Last week I started exploring linting errors within Pandas in further detail. When exploring these errors, I realized that Panda's contributors were causing some of these errors through attempts to better format their documentation, so it is easier to read. For example, one of the errors that came up dealt with "unnecessary indentation", although Panda's program that checks for these types of errors in docstrings identified the indentation as an error, the contributors put this indentation on purpose so the information is easier to read. Therefore, not all linting errors can be easily fixed and require discussion with Panda's moderators to find a better way to write the documentation without the need for such indentation, or keep ignoring some errors in favour of better readability. I will be further talking with moderators to see how we can tackle these issues.

One type of error that I fixed was, "Use only one blank line to separate sections or paragraphs". There were a total of 127 errors that dealt with this. In a massive project like Pandas, it is important to keep documentation and spacing consistent throughout the project. In addition, when the code consists of millions of lines of code, every line that can be removed results in better performance and saves space. My pull request consisted of fixing most of these errors. It was a rather tedious job because I had to go through many files and remove unnecessary blank lines within the docstrings. Sometimes finding these bugs was easy because I had a file and a line number as a reference to the general area of the docstring where the error occurs, however sometimes the lines were not shown or it was trickier to find certain docstrings as they are reused in different files but defined somewhere else.


In addition to the linting errors, I found an issue that got me very interested. The issue asked for the rest of the CircleCI checks to be merged into Azure Pipelines. After the rest of the checks are merged within Azure Pipelines, CircleCI will be removed from the Pandas project. I'm currently exploring different ways to achieve this and will talk in more detail about this issue in the next blog.

Plans for pySearch

The pySearch project has been refactored and restructured by the creator. We agreed that we will not merge any pull requests until this process was completed. So, now that he completed restructuring the project, I will need to go back and ensure that the changes in my pull requests will now reflect the new structure.

Furthermore, the experience that I will receive from my recent issue in Pandas will be helpful for the pySearch project as I will be able to help integrate Continuous Integration like Travis CI, CircleCI, and Azure Pipelines within the project and put some slack on the moderators. With CI, the moderators will no longer have to worry about new pull requests breaking the program.



Comments

Post a Comment

Popular posts from this blog

Another Dropna Bug in Pandas

Image
Another Dropna Bug in Pandas This year I fixed a few bugs that dealt with NaN values and more specifically the Dropna function in Pandas. Every bug like this made me more and more interested in fixing behaviour that is not consistent across the code base, specifically how the functions deal with non existent values. Some functions have a dropna argument. When set to true, the function should drop columns/rows that contain missing values and when set to false, the function should keep columns/rows that contain missing values. Currently, some functions deal well with NaN values, however there are functions that either produce buggy behaviour with NaN or don't deal with it consistently across the code base. This week I found another bug like this and I will be working on it throughout next week. Issue https://github.com/pandas-dev/pandas/issues/10772 The creator of the issue noticed inconsistent behaviour when using the dropna argument inside the crosstab function...
Read more

Unit Tests in Pandas

Image
Contributing Unit Tests to Pandas The issue that I decided to work on this week was one of the easier issues discussed in my last blog, however it gave me valuable experience with unit testing in Python and Pandas specifically. I decided to start my 2019 open-source contributions with unit testing because I haven't had much experience with unit testing previously in Python.  Overview of the Issue https://github.com/pandas-dev/pandas/issues/21205 As I previously stated in my last blog, this issue shed light on an issue with using NaT when converting a Series to a Period using the dt.to_period('D') function. Other datetime64[ns] objects with values have no problem converting into a Period, however when a NaT was passed in, it stayed as datetime64[ns]. NaT represents a missing value for datetime objects. Since the issue was posted, it has been resolved with current updates to the master branch. However, one of the moderators asked for unit tests that would ens...
Read more

Progress in Open Source

Image
Progress in Open Source In this blog I will talk about my journey and progress in open source development so far. I've been really enjoying working on real world projects that are used by many people every day. It is not only challenging as every Pull Request required me to perform research in order to understand the issue and solve it, but it is also rewarding because I acquire hands-on experience that is much needed and also contribute to the open source community that I am very fond of now. Open Source Timeline Summary I started exploring open source development during September of 2018. I had to learn a lot of useful tools that Git and Github offers, which now come in handy almost every day. I now know how to create meaningful issues that follow the specifications of the project, as well as create and structure my Pull Requests to the standards of the project that I'm working on. Furthermore learning how to properly manipulate version history using ...
Read more