Turning a personal Python project into a releasable library












2














I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.



However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.



Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip, and has a stable API.



Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.



So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?



More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.










share|improve this question









New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • jeffknupp.com/blog/2013/08/16/…
    – jonrsharpe
    1 hour ago










  • The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
    – Bart van Ingen Schenau
    25 mins ago










  • @jonrsharpe thanks, there's a lot of super useful information there
    – Nathaniel
    18 mins ago










  • @BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
    – Nathaniel
    16 mins ago
















2














I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.



However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.



Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip, and has a stable API.



Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.



So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?



More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.










share|improve this question









New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • jeffknupp.com/blog/2013/08/16/…
    – jonrsharpe
    1 hour ago










  • The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
    – Bart van Ingen Schenau
    25 mins ago










  • @jonrsharpe thanks, there's a lot of super useful information there
    – Nathaniel
    18 mins ago










  • @BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
    – Nathaniel
    16 mins ago














2












2








2







I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.



However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.



Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip, and has a stable API.



Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.



So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?



More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.










share|improve this question









New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.



However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.



Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip, and has a stable API.



Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.



So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?



More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.







python programming-practices development-process






share|improve this question









New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 3 hours ago





















New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 4 hours ago









Nathaniel

1115




1115




New contributor




Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Nathaniel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












  • jeffknupp.com/blog/2013/08/16/…
    – jonrsharpe
    1 hour ago










  • The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
    – Bart van Ingen Schenau
    25 mins ago










  • @jonrsharpe thanks, there's a lot of super useful information there
    – Nathaniel
    18 mins ago










  • @BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
    – Nathaniel
    16 mins ago


















  • jeffknupp.com/blog/2013/08/16/…
    – jonrsharpe
    1 hour ago










  • The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
    – Bart van Ingen Schenau
    25 mins ago










  • @jonrsharpe thanks, there's a lot of super useful information there
    – Nathaniel
    18 mins ago










  • @BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
    – Nathaniel
    16 mins ago
















jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago




jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago












The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago




The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago












@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago




@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago












@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago




@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago










1 Answer
1






active

oldest

votes


















3














Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.




  1. You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.


  2. Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.


  3. Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.


  4. Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.



Additionally to that:




  1. Make it clear if your library works with Python 2 or 3 or both.


  2. If the library doesn't work on Windows, tell it.


  3. Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.


  4. Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.







share|improve this answer























  • Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
    – Nathaniel
    13 mins ago











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "131"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});






Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.










draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsoftwareengineering.stackexchange.com%2fquestions%2f384748%2fturning-a-personal-python-project-into-a-releasable-library%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









3














Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.




  1. You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.


  2. Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.


  3. Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.


  4. Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.



Additionally to that:




  1. Make it clear if your library works with Python 2 or 3 or both.


  2. If the library doesn't work on Windows, tell it.


  3. Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.


  4. Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.







share|improve this answer























  • Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
    – Nathaniel
    13 mins ago
















3














Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.




  1. You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.


  2. Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.


  3. Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.


  4. Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.



Additionally to that:




  1. Make it clear if your library works with Python 2 or 3 or both.


  2. If the library doesn't work on Windows, tell it.


  3. Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.


  4. Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.







share|improve this answer























  • Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
    – Nathaniel
    13 mins ago














3












3








3






Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.




  1. You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.


  2. Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.


  3. Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.


  4. Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.



Additionally to that:




  1. Make it clear if your library works with Python 2 or 3 or both.


  2. If the library doesn't work on Windows, tell it.


  3. Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.


  4. Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.







share|improve this answer














Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.




  1. You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.


  2. Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.


  3. Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.


  4. Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.



Additionally to that:




  1. Make it clear if your library works with Python 2 or 3 or both.


  2. If the library doesn't work on Windows, tell it.


  3. Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.


  4. Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.








share|improve this answer














share|improve this answer



share|improve this answer








edited 28 mins ago

























answered 35 mins ago









Arseni Mourzenko

113k26279448




113k26279448












  • Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
    – Nathaniel
    13 mins ago


















  • Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
    – Nathaniel
    13 mins ago
















Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago




Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago










Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.










draft saved

draft discarded


















Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.













Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.












Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.
















Thanks for contributing an answer to Software Engineering Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsoftwareengineering.stackexchange.com%2fquestions%2f384748%2fturning-a-personal-python-project-into-a-releasable-library%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Morgemoulin

Scott Moir

Souastre