serious work push
May. 24th, 2013 07:21 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
kareinaI have been gearing up to working longer and more enthusiastically as some major deadlines approach (which was severely interrupted by the Double Wars trip and recovery thereof), and that really showed this week. Yesterday I put in a 14 hour work day, but much of that was fun time spent hanging out with loved ones as they helped me with a project for work.
I think I have mentioned previously that I have LOTS of photos for my samples, and that I am now using a program to keep them organized. Since the program is searchable I have gone to the effort of copying notes about each thin section (including location, list of minerals, what % of the rock is made up of each mineral, how large each is, and any other useful information I might need later) into the "properties" for the album of photos for that thin section. I have been meaning to create a thin section report--with a sheet of paper for each one that has photos and all of those notes. At this point I have only examined 41 of the thin sections (I have 100 now, and plan to make more soon), so it would have been easy enough to create that report by hand in either a word processor or drawing program by copying by hand the photo and the notes into the document. However, if I did that, and I later made changes to the original data I would have to echo those changes in the document, and the document would have required a rather large file size and would, likely, be prone to crashing (bitter experience speaking!)
Therefore I decided that it would make more sense to have a script that would access the database and extract the information I want and automatically generate the report. That way I could re-generate it with little effort every time I changed things or added new thin sections to the database. The only problem with this is that I have zero experience in writing scripts that access a database. So I asked my sweetie for help, and he really got into the project. After trying to help me on my computer he asked me for a copy of the database on his computer, as he much prefers to do this sort of programming on his linux machine. So I copied over the file (luckily, the program has clear instructions in the help file where to find the database itself, and which sort it is if one wants to do something with it from outside of the program itself) and turned him lose with it. While he worked on that I happily did more work filling in the database with information I need and hadn't actually put there yet. Meanwhile we were also hanging out with our friend, L, on line (she is in Poland this week doing collaborative work with some colleagues there), who got in on the project.
Her contribution is an understanding of LaTex, which can be used to take the output from his script and generate the pdf of the report. So I made an example sheet showing what I want it to look like, and gave it to her, and she wrote the code to take the data and return that layout. This was, mostly, easier than his task. It turns out that the database for the photo program was not really designed with accessing information outside of it in mind--it has albums and sub albums, but if you ask for the information on an album then all of the information, INCLUDING its sub albums, show up in one really long line of text. After careful comparison of the text with how the same information looks in the photo he was able to find/modify a parsing script that would break out the information I need. However, the database does not use any sort of standard way of recording blank lines within text entered by the human, so all of the nice paragraph breaks in the data I entered disappears totally in his parsed version, so next he will have to write his own, new, improved, parsing code to get the information out WITH the line breaks. sigh.
We were all having so much fun working on the project that we were still at work after midnight. She was sensible and hung up to go to sleep soon after midnight, but he and I kept going till nearly 02:00. I love it when work projects hold my attention so long!
Unsurprisingly, I woke up this morning thinking of databases, and wondering if I want to actually try to develop a database to hold all of my data,rather than the series of spreadsheets I am using now. I considered that back at the start of the project, but the list of all the different data types is so long and so complexly organized I gave up on the idea. Perhaps if I get that grant approved and the project goes on after November it will be worth the effort to do a conversion from spreadsheets to database.
Now it is time to head to the last choir performance of the semester: the farewell dinner for the exchange students, and we are the entertainment. This makes sense, since about half of our choir is exchange students this term. I hope we draw as many new exchange students next year, or our numbers will be sadly diminished. Luckily the slogan "learn Swedish, one song at a time" is a good one.
I think I have mentioned previously that I have LOTS of photos for my samples, and that I am now using a program to keep them organized. Since the program is searchable I have gone to the effort of copying notes about each thin section (including location, list of minerals, what % of the rock is made up of each mineral, how large each is, and any other useful information I might need later) into the "properties" for the album of photos for that thin section. I have been meaning to create a thin section report--with a sheet of paper for each one that has photos and all of those notes. At this point I have only examined 41 of the thin sections (I have 100 now, and plan to make more soon), so it would have been easy enough to create that report by hand in either a word processor or drawing program by copying by hand the photo and the notes into the document. However, if I did that, and I later made changes to the original data I would have to echo those changes in the document, and the document would have required a rather large file size and would, likely, be prone to crashing (bitter experience speaking!)
Therefore I decided that it would make more sense to have a script that would access the database and extract the information I want and automatically generate the report. That way I could re-generate it with little effort every time I changed things or added new thin sections to the database. The only problem with this is that I have zero experience in writing scripts that access a database. So I asked my sweetie for help, and he really got into the project. After trying to help me on my computer he asked me for a copy of the database on his computer, as he much prefers to do this sort of programming on his linux machine. So I copied over the file (luckily, the program has clear instructions in the help file where to find the database itself, and which sort it is if one wants to do something with it from outside of the program itself) and turned him lose with it. While he worked on that I happily did more work filling in the database with information I need and hadn't actually put there yet. Meanwhile we were also hanging out with our friend, L, on line (she is in Poland this week doing collaborative work with some colleagues there), who got in on the project.
Her contribution is an understanding of LaTex, which can be used to take the output from his script and generate the pdf of the report. So I made an example sheet showing what I want it to look like, and gave it to her, and she wrote the code to take the data and return that layout. This was, mostly, easier than his task. It turns out that the database for the photo program was not really designed with accessing information outside of it in mind--it has albums and sub albums, but if you ask for the information on an album then all of the information, INCLUDING its sub albums, show up in one really long line of text. After careful comparison of the text with how the same information looks in the photo he was able to find/modify a parsing script that would break out the information I need. However, the database does not use any sort of standard way of recording blank lines within text entered by the human, so all of the nice paragraph breaks in the data I entered disappears totally in his parsed version, so next he will have to write his own, new, improved, parsing code to get the information out WITH the line breaks. sigh.
We were all having so much fun working on the project that we were still at work after midnight. She was sensible and hung up to go to sleep soon after midnight, but he and I kept going till nearly 02:00. I love it when work projects hold my attention so long!
Unsurprisingly, I woke up this morning thinking of databases, and wondering if I want to actually try to develop a database to hold all of my data,rather than the series of spreadsheets I am using now. I considered that back at the start of the project, but the list of all the different data types is so long and so complexly organized I gave up on the idea. Perhaps if I get that grant approved and the project goes on after November it will be worth the effort to do a conversion from spreadsheets to database.
Now it is time to head to the last choir performance of the semester: the farewell dinner for the exchange students, and we are the entertainment. This makes sense, since about half of our choir is exchange students this term. I hope we draw as many new exchange students next year, or our numbers will be sadly diminished. Luckily the slogan "learn Swedish, one song at a time" is a good one.