HIPE Advanced Usage Telecon # 1

July 20, 2011


  • Carolyn McCoey (CMC)
  • Mark Taylor (MT)
  • Mihkel Kama (MK)
  • Elena Puga (EP)
  • Javier Diaz-Soto (JDS)
  • Paul Balm (PB, chair)


1. Welcome and meeting goal
2. Summary of presentations at the HIPE Forum 2011 meeting
3. Things that users can currently do (round over observers)
4. Things that users cannot yet do (round over observers)
5. Questions, comments, doubts
6. Next telecon?


1. Welcome and meeting goal

PB states that the goal of this meeting is to provide a forum where users can ask questions and provide feedback and for the HSC to provide answers and to take not of requests from users. The focus of today's meeting is on HIPE features that improve HIPE's usability and on using software from users inside HIPE.

2. Summary of presentations at the HIPE Forum 2011 meeting

PB presents a summary of the following presentations from the HIPE forum: (summaries.pdf):

3. and 4. Round over observers: What can or can you do not in HIPE currently?

CMC: Even to experienced users, a number of the HIPE features from Jaime's presentation are unknown.
PB: Jaime is presenting them at regular intervals at CSDTs/HIPE Forum.
CMC: That way you don't reach all users.
PB: Point taken - let's see if and how we add these to documentation. This will be followed up under SCR HCSS-13743.

MK: Many things have moved in the right direction recently so I will focus on suggestions for improvement:

(MK:1) I think that basic tasks or principle in HIPE are still obscure to beginning users. For example, it should be easy to get a list of observations in pools and storages and to know about the versions of observations. This can probably be achieved by simplifying the Product Browser.
PB: This ties in with a request from Bernhard Schulz at the HIPE Forum. He calls it "the pool explorer", which was agreed should be implemented by simplifying the Product Browser.

(MK:2) There should also be easy buttons to add new pools and storages.
PB: This is currently done from the Preferences (under the Edit menu), which is based on the idea that your set of pools and storages is created once and then you keep what you have-- it's configuration that is done once and then rarely changed. However, we are finding that this is not how people use the system.
People create and remove pools and storages all the time, so probably Preferences is not the right place and this must be moved to a more accessible location. There is no technical difficulty. This has to be taken into account with the simplification of the Product Browser.

(MK:3) The versions of observations are confusing. One would expect that the highest version is the one that is most recently saved, but this is not always the case as shown by the Product Browser. So it's not clear what the version
numbers mean.

TODO Provide an explanation for this. (being followed up under HCSS-13781)

(MK:4) How easy is it to develop for HIPE? I'm generally producing large sets of Jython scripts and I don't have major issues with that. Minor issue: It would be nice to have numpy available. Will be looking into using plug-ins as a sharing mechanism.

EP: If I change the directory where LocalStores are stored (the directory where new pools are created), do I have to restart HIPE? This is inconvenient.
PB: I think that's correct -- you have to restart.
CMC: I concur with EP.
Post-meeting comment: PB tested this and the answer is no, you don't have to restart HIPE. The change is effective immediately. So no follow-up here.

About Topcat and its Jython interface, JySTILTS:
MT: The Jython interface to STILTS allows to do Topcat things like cross-matching, selections on rows, all kinds of things related to tables. Load tables from FITS or VO Table, manipulate, save again, etc. The advantage to not using the GUI (i.e. Topcat) for this work is that you can do it from scripts. JySTILTS and HIPE are now on the same Jython version (2.5) so in principle this can be used. The integration can be improved. (Jy)STILTS does not use TableDataset but StarTables and the conversion between these is inefficient (time and memory consuming).
MK: I have no experience with STILTS or Topcat.
CMC: Me neither, but I do know that Katrina (Exeter) is concerned with tables.
MT: People will have to know it's there to be able to use it.

About sharing work and publishing available scripts:
MK: I'm producing lots of scripts and often find at conferences that other people have done the same scripts. We could be more aggressive about advertising scripts that are available. People might be concerned about getting credit for work they have done but this is not generally an issue.
EP: I've also been unable to find what I need (e.g. the scanamorphos plug-in). I have found useful stuff in ICC toolboxes, contained in HIPE.
PB: There is a web-page with User Contributed Software linked from the main HSC page: http://herschel.esac.esa.int/UserContributedSoftware.shtml
There is also a page with plug-ins, which can be found using HIPE: Tools -->
Plug-ins --> Find more. The URL is: http://herschel.esac.esa.int/twiki/bin/view/Public/DpHipePlugins

We agree that these two pages should at least be linked and perhaps merged, but if they are merged, we can't update the plug-ins page as easily anymore. People are reminded to use the DP interest mailing lists to ask, if they need some general tools. Perhaps Bruno should send a reminder to the lists, that they are intended for such usage.

5. Questions? AOB?

No further questions.

6. Next telecon

Rather than plan a telecon now, it's probably better to circulate a Doodle in 6 weeks or so.