Re: [boost] New Astronomy Library
These days I was studying already available astronomy libraries in
different languages.
I found that "Astropy"( http://www.astropy.org/ ) is well established and
it is being used in many places. Maybe one of the reasons for its
success is that it is for Python. As we all know that python is one of the
easiest languages to learn. Not all scientists are programmers and maybe
that's why they find python and Astropy easy to use.
So what I was thinking is, what if we build a similar structure to the same
library? It will make things easier for those scientists who do not have
much time to learn a new programming language and this will make the
transition easier from python to C++.
Is it a good idea?
Pranam Lashkari
On Mon, Feb 12, 2018 at 7:31 PM, JC Passy
Dear Pranam,
Sorry for taking so long to get back to you. So I talked to some of my former colleagues about potential projects related to the analysis of astronomical data. They forwarded me the following paper which gives an example of the amount of data they are dealing with and some solutions they have found. I think you might find it interesting: http://adsabs.harvard.edu/abs/2017PASA...34...38V
I must say in terms of concrete ideas, they did not have anything specific to suggest. Most of these large surveys have already teams dedicated to develop a pipeline to deal with their specific data. However, we will keep looking and talking to people, may be something will come up in the future.
In the meantime, you seem to already have ideas for amateur astronomers. I suggest you start with it (I am happy to help you if you feel like it) and I will let you know if a potential project for professionals comes up. Does it sound good to you?
Cheers,
Jean-Claude
On 26. Jan 2018, at 12:17, JC Passy
wrote: Dear Pranam,
Sure, I am waiting for some feedback from two former colleagues of mine on that matter. I should get something by early next week, I will let you know as soon as I do.
Cheers,
Jean-Claude
On 25. Jan 2018, at 13:49, Pranam Lashkari
wrote: Dear Jean-Claude;
As you mentioned in your emails, you are having some ideas what can be implemented in this library as per requirement of professionals in this field. Can you start discussing those ideas so that we can get a better idea of how to structure this library?
On Mon, Jan 22, 2018 at 1:40 AM, JC Passy
wrote: Dear Pranam, dear Vinicius,
not sure that my initial email made it to the mailing list. If it did, sorry for repeating myself. I am working as a computer scientist at the MPI for Intelligent Systems in Tübingen. Before that, I did my PhD in astrophysics and a postdoc. In general, I am also interested in using boost and developing a library that could be used by astronomers and astrophysicists.
Looking at your proposal, I would second Vinicius’ opinion in the sense that one should try to make it a bit more specific. In particular, you should emphasize the *why* (what do you wish to accomplish? why would someone need/use it?) and the *who* (who is your target audience?). Regarding the latter, I see two options:
1) *amateur astronomers*: I personally do not know what their needs are, but I could try to find out. There are some open source projects dedicated to amateur astronomers (for instance https://openphdguiding.org to which we contributed), so that could be an option;
2) *professional astronomers*: I think a lot can be done, in particular regarding observational data analysis. They used *REALLY* old tools (IDL, IRAF, …) and given the future and upcoming surveys that will produce very large amount of data, they might have serious performance issues. This is why I believe they might benefit from using cutting-edge C++ libraries.
I would be quite interested in option 2, and it could impact potentially a large audience IMHO. Of course, the “selling” part won’t be easy (people, especially astrophysicists :) won’t give up their old methods easily even though the new method is 10x faster) but we can facilitate this process by pin-pointing features that they really need. I will try to talk to some of my former colleagues this week and see what they think about that. In this case, this is a project that would need a bit of time to be properly defined and the deadline for GSoC18 is coming too soon. As far as I am concerned, this is no problem as this project does not need to be tied to it.
Cheers,
Jean-Claude
On 21. Jan 2018, at 15:00, Vinícius dos Santos Oliveira via Boost < boost@lists.boost.org> wrote:
2018-01-21 8:07 GMT-03:00 Pranam Lashkari
: I have made this proposal for this new library(check attachment) Suggestions are invited.
You need to focus more on "selling your project". Why is it important to have such library in C++ specifically? Isn't this one of those projects where other languages will do just fine? What market are we missing by not having this library? What C++ have to offer to make such library be inviting to write in C++?
You mention "gather information from observation data". Does this mean you'll include a parser for some format?
Can you compare this library with some non-C++ library and detail a little how will scope between the two differ (e.g. your library will focus initially only on X given the library is only starting)?
For now, this should be good. When it's time to submit the student form through the GSoC platform, you'll also focus on selling yourself. I'll give more feedback on this when the time comes. Don't worry.
Should I publish this on the wiki page of Boost GSoC 2018?
Yes, please. You can put my name as a potential mentor.
-- Vinícius dos Santos Oliveira https://vinipsmaker.github.io/
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
On 13 February 2018 at 05:45, Pranam Lashkari via Boost < boost@lists.boost.org> wrote:
... As we all know that python is one of the easiest languages to learn. Not all scientists are programmers and maybe that's why they find python and Astropy easy to use.
The above is being repeated over and over and over again. I find python anything but easy to learn, it's rather confusing actually (and verbose). If you would want an easy (a little over 20 keywords), versatile language (with some rather advanced ideas), pick Lua.
... build a similar structure to the same library?
Isn't it a python lib using NumPy? You're gonna provide language bindings?
It will make things easier for those scientists who do not have much time to learn a new programming language and this will make the transition easier from python to C++.
If you're a scientist, I doubt you'll decide to move from Python to C++ (or C for that matter)... Tensorflow has a C++ API, it's so underused that the documentation of that API is missing many bits. If you cannot program, trying to do that in C++ is gonna be a very frustrating experience (as in not making any progress whatsoever, which means no research grants), iff you have to start from scratch.
Is it a good idea?
Hard to say. Maybe you could explain (to the community) what you would like to achieve in terms of "deliverable(s)", i.e. what will the end result be (as in, what kind of thing). The issue is maybe/possibly that Astronomy, as a field, seems to be rather niche for a general library as Boost. degski
Sorry for taking too long to replay as I was talking to one of my friends
who is a scientist and works in an observatory.
I completely agree with you that scientist would not move from Python to
C++ easily. But Astronomical data are getting bigger in size and Python
takes too much time to compute them (according to my scientist friend).
That's where C++ can be a dramatic improvement.
Coming to the point of the structure of the library, when I said that we
can follow the structure of Astropy I meant "class structure"(class
hierarchy) and API.
*Now the point is what we can do with this library:*
In observatories to make day to day programming easy, these are some
specific topics which can be covered here,
1. Noise reduction in astronomical data(various algorithms)
2. Galaxy morphological classification(classification based on visual
appearance)
3. Comparison of images(To find the motion of moving object in space)
4. Object detection
5. Light curves of variable stars
6. Spectroscopy
Above mentioned topics were suggested to me because these are used on daily
basis and can make things faster.
now coming to the amateurs
1. the coordinate system can help to plan out observations.
2. reading data from a file(astronomical catalogs)
3. Creating observation lists
*So the ultimate goal of this library would be to make day to day astronomy
easy and faster by providing features that are used on daily basis by
professionals and amateurs.*
thank you.
pranam lashkari.
On Tue, Feb 13, 2018 at 7:39 PM, degski via Boost
On 13 February 2018 at 05:45, Pranam Lashkari via Boost < boost@lists.boost.org> wrote:
... As we all know that python is one of the easiest languages to learn. Not all scientists are programmers and maybe that's why they find python and Astropy easy to use.
The above is being repeated over and over and over again. I find python anything but easy to learn, it's rather confusing actually (and verbose). If you would want an easy (a little over 20 keywords), versatile language (with some rather advanced ideas), pick Lua.
... build a similar structure to the same library?
Isn't it a python lib using NumPy? You're gonna provide language bindings?
It will make things easier for those scientists who do not have much time to learn a new programming language and this will make the transition easier from python to C++.
If you're a scientist, I doubt you'll decide to move from Python to C++ (or C for that matter)... Tensorflow has a C++ API, it's so underused that the documentation of that API is missing many bits.
If you cannot program, trying to do that in C++ is gonna be a very frustrating experience (as in not making any progress whatsoever, which means no research grants), iff you have to start from scratch.
Is it a good idea?
Hard to say. Maybe you could explain (to the community) what you would like to achieve in terms of "deliverable(s)", i.e. what will the end result be (as in, what kind of thing).
The issue is maybe/possibly that Astronomy, as a field, seems to be rather niche for a general library as Boost.
degski
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost
Hi Pranam,
On 22. Feb 2018, at 15:14, Pranam Lashkari via Boost
wrote: Sorry for taking too long to replay as I was talking to one of my friends who is a scientist and works in an observatory.
I completely agree with you that scientist would not move from Python to C++ easily. But Astronomical data are getting bigger in size and Python takes too much time to compute them (according to my scientist friend). That's where C++ can be a dramatic improvement.
I am a scientist and I work in the field of (astro)particle physics. I also happen to know one of the leading developers of gammapy and astropy. I think that "Python takes too much time to compute" is not a good argument for your project. Most of the computation in packages like astropy and gammapy is based on numpy which is very fast. And where it is not fast enough, people who already use Python will rather use acceleration techniques such as Cython and Numba. C++ is a hard language, difficult to learn and almost impossible to master. As a scientist you want to spend a maximum of time doing science, and a minimum amount of time understanding computer code and the lovely error messages that template-heavy code produces. Also keep in mind that science is about short-term projects with a high fluctuation of personell. The ease of using and learning Python in this context greatly outweighs potential performance issues. As a project leader, you don't want to spend many months getting the new people up to speed with tricky software. You would clearly prefer Python solutions over C++ solutions whenever possible. In summary, no, I don't think that a C++ astronomy library in Boost will be a hit. I recommend you join forces with astropy, ask them which algorithms are slow, and try to speed them up by writing them in C++, using Boost.Python or pybind11 for the Python bindings. Best regards, Hans
Yes maybe you are right.
This idea does not seem promising for the community so i should drop this
idea here and focus on something better.
Thank you.
Pranam Lashkari.
On 22-Feb-2018 10:25 PM, "Hans Dembinski"
Hi Pranam,
On 22. Feb 2018, at 15:14, Pranam Lashkari via Boost < boost@lists.boost.org> wrote:
Sorry for taking too long to replay as I was talking to one of my friends who is a scientist and works in an observatory.
I completely agree with you that scientist would not move from Python to C++ easily. But Astronomical data are getting bigger in size and Python takes too much time to compute them (according to my scientist friend). That's where C++ can be a dramatic improvement.
I am a scientist and I work in the field of (astro)particle physics. I also happen to know one of the leading developers of gammapy and astropy.
I think that "Python takes too much time to compute" is not a good argument for your project. Most of the computation in packages like astropy and gammapy is based on numpy which is very fast. And where it is not fast enough, people who already use Python will rather use acceleration techniques such as Cython and Numba. C++ is a hard language, difficult to learn and almost impossible to master. As a scientist you want to spend a maximum of time doing science, and a minimum amount of time understanding computer code and the lovely error messages that template-heavy code produces. Also keep in mind that science is about short-term projects with a high fluctuation of personell. The ease of using and learning Python in this context greatly outweighs potential performance issues. As a project leader, you don't want to spend many months getting the new people up to speed with tricky software. You would clearly prefer Python solutions over C++ solutions whenever possible.
In summary, no, I don't think that a C++ astronomy library in Boost will be a hit. I recommend you join forces with astropy, ask them which algorithms are slow, and try to speed them up by writing them in C++, using Boost.Python or pybind11 for the Python bindings.
Best regards, Hans
participants (3)
-
degski
-
Hans Dembinski
-
Pranam Lashkari