Category Archives: Methods to assess TEM and AFM images

GIMP for processing AFM images – unsharp mask

There is a dilemma for researchers in terms of their choice of image processing programs.  The industry giants charge considerable amounts (matlab is exorbitant), what used to be high prices for CDs but now for online subscriptions.  Does the US taxpayer need to pay for these services – it doesn’t seem quite right that we pay industry graphics prices to do science which benefits all of humanity (hopefully).

I have spent the better part of the year (not full time but avidly) trying to figure out who has the image processing program that is the easiest to use (i am talking here about real ease of use…. as I come to this field as a visual learner, without apologies, but find much lacking in the so called “intuitiveness” of many programs). I am also interested in the least expensive program as i dont want to feed the giants in the commercial world.  Another goal is to see whether all the bells and whistles that are touted in the new image processing programs really make a difference for scientists who are wanting to do some relatively simple things….  get rid of dirt, get rid of pixels, identify and enhance hard to see details, to put statistics ahead of guessing about shapes, peaks and the like.

I began 50 years ago with wet-darkroom prints, and the simplest tablet and stylus, haha i don’t even remember the name of the setup but could look that up in a materials and methods published somewhere.  I used camera lucidas and pencil, then digital images.  The earliest processing was just a hand waving  (dodging and burning) then using those same terminologies, photoshop appears.

The earliest plates for publication were made on matboard with a drymount press and tissue.  The earliest program that i used for making images into figures was the first version of CorelDRAW (it was version 3) cracked on 11 pink 3.25 inch floppy discs.  These were mostly plots and line drawings.

50 years later, I still use CorelDRAW (x5 and 19), CorelDRAW Photopaint (x5 and 19) and still use Photoshop (6 and 2021), and have in the past year examined ImageJ, Gwyddion, and GIMP.  It is difficult not to be biased because CorelDRAW has had both raster and vector elements that in my way of thinking are more than adequate for both processing images, measuring features, and creating diagrams. CorelDRAW has been nicely compatible with Photoshop and Illustrator (though the reverse has never been as smooth). I have used images with 100 layers or more imported into CorelDRAW from Photoshop with ease.

So the purpose of this is to determine whether it is possible to do image processing with the same filters and measuring options in the open source software as is done with the high priced industry standards.

The test today is the unsharp mask filter in GIMP.  I do like the ease with which the sliders can be manipulated with instant preview updates….  i don’t much like the actual “typing” from the number pad into the slider.  But using the surfactant protein D images (that I have used from published papers (in this case those shown here are some good and not so good images from Arroyo et al  (AFM of surfactant protein D).  Regardless of the original image quality, GIMP (open source image processing software) does a great job of masking.

Pay special attention to the small masked areas in the collagen like domain of each of the SP-D trimers….  yep…. 3 to 4 small peaks beside the glycosylation peak, especially nice in lower right image, and in that one and the lower left image… a tiny peak on the side of the N termini peaks is a new little peak (of which i am trying to determine the relative incidence).  Error bars are what i have used for measurments and are visible, most are 100nm but the tiny one is 20nm, and lower center has no error bar.

 

Beautiful SP-D image

This image (original from supplement of Arroyo et al) was first a screen print, then in gwyddion i added a 10px gaussian blur, then stretch color range >175 to 255.  This virtually eliminated background, but at the same time decreased the height of the various peaks.  The upside is that there is a rotational “look” to each of the trimeric arms, and there is also the hint of the tiny proximate peak to the N termini which shows up just before the large glycosylation peak on each trimer.  Clearly too are the small rhythmic peaks along the collagen like domain (almost always 3 or 4 in descending prominence and width).  One can rule out any bumping and noise since all these dodecamers fall at different directions and so scan lines are an annoyance but contribute to peak height on a random basis. CRD are show as multi-lumpy units.  Looks like the alleged glycosylation peak is present on all four trimers pretty equally.

 

-Perfect view of what I think will pan out is shown in the lowest arm (vertical hexamer) which shows the Nterm, the tiny adjacent peak, the glycosylation peak, three peaks along the collagen like domain and the neck and CRD in a raindrop shape.  Pretty awesome.

 

Tone curve in corel photo paint: I dont think it is possible to erase the data

Doesnt seem to matter much what processing happens to these images. The data remain abundantly clear for SP-D dodecamers. N termini, glycosylation peaks and three small peaks in the collagen like domain and nice CRD.  At this point: it doesn’t matter the direction, or the lines in the AFM, the image processing, or the angles between the trimers — it looks like three small peaks in the collagen like domain. Over and over again they show up.  See tone curve graph in bottom image. SP-D imagesaved and processed from Arroyo et al.

Image processing is fun sometimes: POSTERIZE

Image processing is fun sometimes with good results.  This is “posterize” after other manipulations like gaussian blur and HSL and BCI adjustments.  CorelPhotopaint19 processed images just for pleasure provide the same data as the routine microscopic (approved more or less) processing algorithms.

It is easy to see the intense brightness of the glycosylation sites (not always 100 percent in all the molecules (presumably the most intense when all three molecules in the trimer are glycosylated, but visible in a lesser amount where maybe one or two are) but so nicely shown adjacent to the N termini junctions.  The CRD are just great, and the very consistent peaks in the collagen like domain of each trimeric arm just are verified.

Whats missing here are the tiny peaks on each of the four trimers small and very close to the N- peak.  Enjoy. (thanks to arroyo et al, as usual)

 

Photoshop 6 vs photoshop 2021: looks like no difference

Image processing is helpful in interpreting some types of photos.  It doesn’t seem to be of much help with traditional TEM at all, in terms of sharpening or moderating the images outside of some very basic hue/saturation/lightness options, basic contrast, and getting rid of some pieces of dust dirt or previous labels.  Shadow cast and negative stained image with TEM are a little bit of a different story and some processing can help, but the biggest place that I see processing images (not the big and aggressive processing like masks and segmentation and counting objects) to remove pixelation, or increase the readability of LUT plots, is in AFM.

One sort of sad thing is that companies that are the best at using unique algorithms for manipulating raster images are also very expensive, and have invasive (in you face computer antics to which i can attest from the recent owners of CorelDRAW and of Photoshop 2021) and may not provide much if any improvement from the old programs which one typically has on disc.

In trying to determine the number of  brightness peaks in the trimeric arms of SP-D I began to feel guilty about using imaging programs that were 10 years old and decided to compare image processing outcomes of newer versions of the two programs i use most often (CorelDRAW and Photoshop).  In this endeavor I also tried the processing of ImageJ and Gwyddion and found that they are not really preferable to the general run of the mill old versions of CorelDRAW and Photoshop.  Image here compares an original photoshop 6 (easily 10 years old) and photoshop 2021 (a subscription) (the same processing was used for both images  (the original image of SP-D derived from a screen print of Arroyo et al, as mentioned countless times now).  Photoshop 6 on the left (green plot lines) Photoshop 2021 on the right (red plot lines), and arms 1 and 2 digitized in ImageJ and each similar arm (as arm 1 (left) and arm 2 (right) plots superimposed. The processing in this case was the same in both photoshop versions, that was  image adjustment>HSL (with hue not changed, saturation-25 and lightness -40)(written as HS-25L-40).

Bar maker from the original image was kept with each figure (and was reported in this image as being equivalent to 20nm). Diameter of each dodecamer digitized in ImageJ and touches the outer most edge of 3 of the 4 dodecamers (in this particular image things were pretty tidy and all four CRD to come close to the circumference of the circle, but this is not usually the case). Background is in set of plots on right.

Segmented line tracings of each hexamer began thusly: arm 1, most left portion to the right hand (near center horizontal) arm 2, begin top slightly right of center and descend to bottom of the hexamer.  This can be checked by the peak that has two close bumps on the left side of the vertical plot where two very close bright spots can be seen.

The objective of this whole exercise is still to determine how many peaks are in the area of the collagen domain between the alleged glycosylation site and the neck+CRD. Hopefully along with that information is some reasonable insight into what processing programs are easy cheap and sufficient, and what algorithms enhance the data without changing it.

Original screen print from supp Fig 4 is here.

Comparison of LUT plots with different image processing programs

Comparison of LUT plots with different image processing programs has morphed out of my desire to determine how many peaks are along the collagen-like domain of SP-D dodecamers.

I have used half a dozen or more processing programs and many many variations on blurs and medians and sharpens and masks and there is great consensus in the shape of the LUT plots so it is pretty easy to suggest that what looks nice, when compared to an original plot is pretty much going to reflect what is actually present in the image.  Case in point in the figure below where an image (screen print from the actual manuscript (oft used Arroyo et al) processed in coreldraw x5 vs photoshop 2021 really show astounding similarity. The differences in the peak heights are a result of HSL and contrast maniuplations but this doesnt change the data. Diameter of the dodecamers is in inches derived from the original images, and must be reasured against the original bar marker dimensions of 20nm given in the original publifcation (0.69 inches relative to the 5.22 here) making the diameter of this particular dodecamer around 150nm.

Amazing transformation with noise and blur filters

I thought this was a rather unlovely picture of a dodecamer which i cut and pasted from a pixelated image which was part of a SP-D publication and a figure from Hartshorn et al. The original crop and imageJ LUT plot is shown along with the images processed in ImageJ (filters>unsharp mask> 20 px radius, 50% mask  and then 10 px gaussian blur).  The smoothness of the LUT plot was greatly enhanced, and the number of peaks in this seemingly unhelpful original image turned out to be very close to the LUT plots that have been obtained for Arroyo et al’s AFM images of SP-D.

Middle and bottom image left = processed image and trace lines in ImageJ. Plots from the unprocessed and processed images superimposed.  They are basically identical but the processed images is smooth.  In this particular dodecamer the relative difference in the N termini peak height and the glycosylation peak on either side, if one were to theorhetically divide the N termini peak into quarters, a peak this size might mean that the glycosylation of only a single molecule of the trimer is glycosylated or none. ( I have not seen this proposed before, but peak height at the site of the glycosylation site should be quantifiable and relate to the amount of glycosylation.  Does anyone have an opinion on why one would expect glycosylation to be an all or none phenomenon – that is, all three molecules in the trimer? does anyone entertain the idea that it could be on a scale of 1-12?

I am really pleased to see a consistent 3 peaks on either side of the N termini, with just a hint of the new peak that shows up very near the margins of the N termini peak. (see previous posts and pictures on this site).

Either 1 or 2 peaks can occur in the CRD section of the molecule.  so the two peaks at the left hand side of all four plots seems to be related to the head and neck domains and the irregularity of their positions in most micrographs.

 

Adjusting arms of SP-D dodecamer to find LUT peaks

Adjusting arms of SP-D dodecamer to find LUT peaks — not a new thought but one that could help determine how often there are 3 vs 4 peaks between the N termini peak and the LUT peaks for the CRD.  This image is from a publication by Hartshorn et al, and there was a composite image that had about 15 dodecamers of SP-D that I am measuring for:

1 hexamer length, 2) number of peaks, 3) level of glycosylation (subjective measure of potentially 12 sites in a dodecamer), 4) comparing brightness of the alleged glycosylation peak in each of the four trimers 5) using all the SP-D AFM and TEM images I can find to get a general size (i think it is very close to 131nm (and this value is hundreds of measures, and includes errors by individuals that made the images, mainly in bar markers, or missing bar markers, and by errors in how i remeasured those markers and in the inherent errors in tracing with a segmented line the distance in ImageJ etc). Image on the top, unprocessed and a screen print from a maximally enlarged image (enlarged to the identical degree, accompanying bar marker). No anti alias on save, therefore it is pixelated.

Middle plot is a trace of both hexamers, no adjustments, just the same size graph superimposed, along with a tracing of the background noise (orange bar).

Bottom plot is adjusted by using the N termini peak in both tracings and placing that in the “center” of the plot. This compensates for some of the stretching that has occurred on the right hand side of the dodecamer during processing.  It does center better the peaks along the collagen like domain, and includes a better positioning of the alleged glycosylation peaks. This particular dodecamer was very off center but after controlling for that distortion, shows similarity to other dodecamers.  Count by eye and count of the peaks found in the plots along each hexamer  were 10 (by eye), and 16 (red plot on middle graph) and 10 (by eye) and 19 (blue plot line on middle graph).  Clearly the eye accommodates some of the pixelation while the plot does not.

My hope is that all the plots of all the dodecamers from all the publications that show images will be used at some point by someone who knows signal processing will determine with accuracy how many peaks really do exist and their relative heights.

 

Looking more carefully at glycosylation sites on SP-D dodecamers

There is quite a bit of variation in the heights of the LUT peaks plotted in SP-D dodecamers (same is true visually with other oligomers, but i have not measured them yet). In the summary sheet I created with 80+ images of SP-D visualized by AFM in publications just by Arroyo et al, it seems pretty clear that not all the molecules in all the trimers are glycosylated.

It is easy to see some distinct brightness on either side of the Ntermini junction by eye, after processing, and easily seen on LUT plots (in fact to miss it is not really possible), but it is not consistent enough in height but is consistent enough in place) to allow some speculation that each of the trimers might have a different amounts of glycosylation.

This is also apparent in the single image from these manuscripts that shows a de-glycosylated dodecamer and there is inconsistent “brightness” in those first peaks on either side of the Ntermini peak.

Below is the same multi-image paste up or layout with about as many dodecamers (there is one pentadecamer and one nonamer excluded from the calculation) as i could easily dissect from the figures. Each has my quick assessment of brightness (peak height) at the site that they call the glycosylation site.  If there was no peak, then 0, if there was an “iffy peak”, then 0.5  if clearly all four trimers in a dodecamer had a bright peak, that was a 4.

Mean 2.6 arms glycosylated,  median 2 peaks, mode 2 peaks n=83.  I have not yet found a specific statement that says one or two molecules in a trimer can be glycosylated, or even just one.  Larger image is two or three posts back.

LUT plots of SP-D hexamer-arms

LUT plots of SP-D hexamer-arms from SP-D dodecamers. These, as mentioned many times, are from publications by Arroyo et al, and my purpose has not changed. Look for consistent peaks along the collagen like domain, and to sort out which processing algorithms are useful for analyzing AFM and TEM images.  At this point I think that image processing is cosmetic. It does make plots more pleasing but really doesn’t change the peaks much. (I am not talking about the fantastic and artistic filters that something like CorelPhotopaint and Photoshop2021 can offer….  but the simple filters that remove scars, noise, pixelation but don’t really change data.

Plots and image below: two relatively matching dodecamers of SP-D found hunting through the sometimes crowded and pixelated image that are found in publications. the left hand image is purported to have a glycosylation peak (next to the brightest peak in the center of each hexamer (the N termini). I am not convinced that all arms of a dodecamer are glycosylated to the same extent, surely it is easy to see that peak height is not the same on the four trimeric arms.  The image on the right is a de-glycosylated SP-D that just happens to conform to the same X style as the image on the right.  THis is not the most typical form since angles between the trimers are most often acute and obtuse, not right angles as seen here…. but this is a good spread and can show the peaks along the arms nicely. Image on the righ has diminished peaks on the horizontal plot, but does have a peak on the vertical plot lower half (see white arrows where peaks should not occur in completely de-glycosylated SP-D but do seem to show some peak).  All four arms of the SP-D have differing size peaks (red and green arrows).

The filter used here is a “highpass” (a filter option present in the four programs that i have most often used to look at image processing as an aid to molecular modeling (CorelDRAW x5 with attending CorelDRAW photopaint, CorelDRAW19 and attending CorelDRAW Photopaint as well as Photoshop 6, and Photoshop 2021. These programs are touted to be about the best around.  I have used Gwyddion as well, but find the ease with which the former can be used is a time saver.  ImageJ is an easy program (and i have not found an equivalent in photoshop or CorelDRAW or Gwyddion) for making LUT plots.

Red plots (most horizontal arms, green plots, more vertical arms — these formations are by chance). At least this pair would suggest that level of glycosylation is graduated, and maybe not present through out a dodecamer the same on each arm.

Just in general…. looks like a minimum (including the glycosylation peak) of four bright areas occur along the collagen like domain.  This is so consistent that I am hopeful to get statistics to verify this.