Turning a personal Python project into a releasable library
I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.
However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.
Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip
, and has a stable API.
Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.
So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?
More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.
python programming-practices development-process
New contributor
add a comment |
I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.
However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.
Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip
, and has a stable API.
Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.
So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?
More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.
python programming-practices development-process
New contributor
jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago
The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago
@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago
@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago
add a comment |
I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.
However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.
Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip
, and has a stable API.
Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.
So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?
More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.
python programming-practices development-process
New contributor
I'm an academic rather than a programmer, and I have many years' experience writing Python programs for my own use, to support my research. My latest project is likely to be useful to many others as well as me, and I'm thinking of releasing it as an open-source Python library.
However, there seem to be quite some hurdles to cross in going from a functioning personal project to a library that can be installed and used painlessly by others. This question is about the first steps I should take in order to start working toward a public release.
Currently, I have a single git repository that contains my code that uses the library as well as the library itself, and I use git as an emergency undo button in case anything breaks. All of this works fine for a single user but is obviously not appropriate if I want to release it. Where I want to end up is that my library is in a separate repository and can be installed by others using pip
, and has a stable API.
Learning to use setuptools etc. is probably not so hard once I'm at the point of wanting to publish it - my problem is in knowing how I should be working in order to get to that point.
So my question is, what are the first steps one should take in order to start preparing a Python library project for public consumption? How should I reorganise my directory structure, git repository etc. in order to start working towards public a release of the library?
More generally, it would be very helpful if there are resources that are known to be helpful when attempting this for the first time. Pointers toward best practices and mistakes to avoid, etc., would also be very helpful.
python programming-practices development-process
python programming-practices development-process
New contributor
New contributor
edited 3 hours ago
New contributor
asked 4 hours ago
Nathaniel
1115
1115
New contributor
New contributor
jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago
The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago
@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago
@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago
add a comment |
jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago
The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago
@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago
@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago
jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago
jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago
The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago
The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago
@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago
@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago
@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago
@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago
add a comment |
1 Answer
1
active
oldest
votes
Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.
You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.
Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.
Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.
Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.
Additionally to that:
Make it clear if your library works with Python 2 or 3 or both.
If the library doesn't work on Windows, tell it.
Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.
Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "131"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsoftwareengineering.stackexchange.com%2fquestions%2f384748%2fturning-a-personal-python-project-into-a-releasable-library%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.
You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.
Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.
Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.
Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.
Additionally to that:
Make it clear if your library works with Python 2 or 3 or both.
If the library doesn't work on Windows, tell it.
Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.
Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
add a comment |
Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.
You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.
Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.
Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.
Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.
Additionally to that:
Make it clear if your library works with Python 2 or 3 or both.
If the library doesn't work on Windows, tell it.
Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.
Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
add a comment |
Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.
You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.
Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.
Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.
Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.
Additionally to that:
Make it clear if your library works with Python 2 or 3 or both.
If the library doesn't work on Windows, tell it.
Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.
Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.
Adding a setup.py, while necessary, is not the most important step if you want your library to be used. More importantly is to add documentation and advertise your library. Since the second point strongly depends on the library, let me rather focus the documentation aspect.
You know everything about your library. And this is problematic. You already know how to install and how to use it, so many things may seem intuitive or plainly obvious to you. Unfortunately, the same things may be neither obvious, not intuitive for the users. Try to look at your library as if you knew nothing about it, and more importantly, ask other people to use it and try to spot all the difficulties they had.
Explain, in plain English, what is your library about. Too many libraries assume that everybody knows about them. When this is not the case, it may be difficult to grasp what is the purpose of the library.
Write detailed technical documentation, but also don't forget about short pieces of code which show how to do some of the tasks with your library. Most developers are in a hurry, and if they need to spend hours trying to understand how to do a basic thing, they may tend to switch to other libraries.
Include your contact information. If your library is a success (and my own experience have shown that this is the case even for rather unknown ones as well), people would encounter difficulties with it: either bugs or simply difficulties understanding or using some parts of it. It is often useful to receive their feedback to improve your library: for every person who reported a problem, there are possibly hundreds who, when encountering it, would just prefer to switch to another library.
Additionally to that:
Make it clear if your library works with Python 2 or 3 or both.
If the library doesn't work on Windows, tell it.
Ensure you use official conventions (use pep8 to check). If not, either explain it clearly or fix it.
Take care of handling edge cases. When your library is called with a wrong type or with a value which is not supported, it should tell, in plain English, what exactly is wrong. What it shouldn't do is to raise a cryptic exception ten levels down the stack and let the user figure out what went wrong.
edited 28 mins ago
answered 35 mins ago
Arseni Mourzenko
113k26279448
113k26279448
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
add a comment |
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
Thank you, I completely agree that the quality of documentation makes or breaks a project. (It's usually the second thing I check when deciding whether to use a project, after the date of the last commit.) On a more technical level, there is a confusingly large ecosystem of tools for managing documentation of Python code. How can I tell which one I should invest in learning for my project?
– Nathaniel
13 mins ago
add a comment |
Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.
Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.
Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.
Nathaniel is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Software Engineering Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsoftwareengineering.stackexchange.com%2fquestions%2f384748%2fturning-a-personal-python-project-into-a-releasable-library%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
jeffknupp.com/blog/2013/08/16/…
– jonrsharpe
1 hour ago
The best way to check if a library is ready for release "into the wild" is to ask a fellow researcher or a student to try to use it and to write down all the difficulties they run into. If they can use it without constantly having to call you for assistance, then the library is in a shape that it can be used by others.
– Bart van Ingen Schenau
25 mins ago
@jonrsharpe thanks, there's a lot of super useful information there
– Nathaniel
18 mins ago
@BartvanIngenSchenau thank you, I'll definitely bear that in mind once I'm close to that step. I'm very much at the "first steps" stage now, of taking something that works but is very far from ready for release, and wondering how I should be doing things now to make sure it can become releaseable in the future.
– Nathaniel
16 mins ago