Clustering items by tag

I write notes in axpress, my still private web front end to a triple store. The notes have a body and a set of tags. Most notes are thoughts about problems I encounter while programming, explanations of design decisions. They have come in handy as a reference a few times, but the information in them is often hard to find, even with tags. I also find writing tags tedious. I’ve decided to look through the various set of tagged items, my notes and my bookmarks to start, to see what kind of patterns I can find.

I used graphviz to get a quick visualization of the relationships between each tag. Each tag is a node. Each node is labeled by the tag name and the number of notes it appears in. The closer two nodes, the more often they occurred in the same notes. Only tags which were connected to more than 1 other tag were included in the visualization to reduce the complexity.

From here, some basic patterns can be seen.some basic subgraphs can be seen. “trade”, “local” and “books” all stem from “economy” only. “economy” is probably a good tag to classify that group of notes where “trade”, “local” and “books” are sub tags. A similar pattern can be seen coming from the express tag (which is the old name of this project). Coming from it are some tags which I don’t seem to note any more.

Only visualizing pairs of tags which occurred more than 3 times, show which (common) tags are used with which other (common) tags, but not how often. It is interesting to see which tags are independent of others. “axpress” and “express” do not occur together often. It might be possible to use this type of graph to find a set of tags which partition the entire set of notes efficiently. This, hopefully small, set of tags could be used as a starting point for drilling down to find a note you are looking for.

For this next graph I wanted to see how ‘thick’ the connections between each node was. That is, from “axpress” how many links are there to “case”? How many to “design”? Are there any tags which occur with nearly every instance of “axpress” or are they uniformly distributed? The darker a node is, the more occurences of that tag there are (color corresponds linearly to number in the label). The color on the end of each link correspond to the relative number of connections represented by that link. For example coming from “code” is a dark green node to “axpress” and a lighter cyan to “case”. This is because there are twice as many connections from “code” to “axpress” as to “case”. From most tags, it seems that there is a relatively uniform number of connections to each other tag.

This set is rather limited, so I am considering a few different next steps. I could use the text of the note in a similar kind of analysis - increasing the number of tags, leaving the number of tagged items alone. Or I could use data from delicious (either my own, or a small set of users) to increase the number of tagged items while leacing the number of tags relatively small. What would you like to see done next? Something else entirely?

CEB Wheelbarrow

We have multiple wheelbarrows for moving blocks from the slide to the pallets where we store them, but this one is the easiest and lightest. The bucket of a standard wheel barrow was removed and these pieces of wood were screwed on in its place. I’m not sure it would be worth removing the bucket from an otherwise functioning wheelbarrow, but this one is definitely easier to use than the others.

CEB Wall Weep Holes

I’ve received new information and so have updated the post. Quoting Dan Johnson

It can be expected that a standard brick wall will have water penetration from weather on the the exterior and possibly condensation on the inside. Many wall designs today have a brick veneer- space for drainage-rigid foam- framing,cmu, etc. For this reason weep holes are used to allow the water a point of exit. However, a CEB wall that is plastered on the exterior or even covered with another type of cladding, should not have that issue so no need for weep holes. If the wall was left unplastered/exposed then it could have that issue and you would need weep holes but I would not recommend that for a cold rainy climate unless you had a really well stabilized block.

Old (Incorrect) Post:

If water pools on the foundation and soaks the bricks, they will loose their structural integrity. We need 1/4” ID copper pipe and cotton rope inside every 4-6ft between the foundation and the first course of bricks as weep holes. The bricks are 6” wide and if weep holes are installed every 4 feet along all 94’ of the building’s outside perimeter that means we’ll want 24 weep holes. 24 holes * 6” deep holes → 12’ of copper pipe and cotton rope.

CEB Tarping

We have been making CEBs one weekend and then laying them over the next few weekends, which means we needed a way to safely store them. We stack them on pallets and then cover them with tarps and metal roofing. Our tarps are old and have some small holes, and if that hole happens to be at a gap in the blocks, a valley will form that collects and drips large amount of water to the blocks below. The metal roofing protects against the rain and the tarps ensure that the blocks dry slowly, which is good for their strength.

CEB Press Slide

I couldn’t find any cheap conveyors around here that had rollers that were close enough together to work for the CEBs. I found and bought one conveyor, but the rollers are too far apart, and so the blocks bounce around rather than roll smoothly. Instead of spending a hundreds or thousands of dollars for a new conveyor, we built this slide with a large 2x12, some 2x2s and some stainless steel flashing, and it has been working well:

Concrete blocks, CEBs and wood hold the slide up at the right height. A heavy block is placed at the end of the slide to keep it from getting pushed away from press when it gets full. We had a few accidents where the whole slide fell over when it got really full and we didn’t have a weight at the end.

CEB Press Hydraulic Pressure Switch

Just spent an hour or two looking for low cost pressure switches to use on the CEB Press. I would like to add them as a way to add reliable sensing of end cylinder stopping points. This avoids the need for external sensors to determine when to stop primary and secondary movements. I will still need sensors to know when the primary cylinder should stop growing the compression chamber as well as one to know when the secondary cylinder should stop for a compression stage. However, with just 2 sensors instead of 6, it should be a fair bit simpler to use physical switches instead of the unreliable hall effect sensors.

The sensor I found to use is this one. It is $32 adjustable from 1000-3000 psi and rated for maximum 6000 psi overload.

There is one at the surplus center here, but the one they sent me was used and broken, there is no datasheet that exactly matches what they sell and there are only 3 left in stock which means that they aren’t going to be able to be used by many more people. I’d rather find a solution that can be easily replicated.

CEB Press First Bricks


We’ve made 15 bricks with our CEB Press so far. We’ve tried a few different mixes:

first layer clay soil only
first layer clay soil only with 2 quarts water per 5 gallons of soil
subsoil clay

Note that we’ve been in a pretty serious drought lately, so the soil is all very dry. The subsoil was more moist than the first layer of clay. The subsoil started about 2 ft down and was more red in color than the first layer.

All of the bricks have minor cracks in them, though the ones with added water are worst, as are the ones which were dried in the sun. One of the first layer clay only bricks was dropped from increasingly higher distances until it broke in half at a 5ft fall after 2 days of curing. It appeared to be more dry on the outside surface than on the inside, which leads me to believe
that it would have been even stronger had it cured for longer.

The shaker motor was tested and seems to shake the machine quite well. Still need to work on the controller board to get all solenoids operable and automatic.

I looked into buying a Power Cube for the work party, but it doesn’t seem like it will be able to be manufactured in time. On the other hand, we may get away with renting only one tractor which will both till and load soil into the CEB hopper. That leaves Danny’s parents’ tracktor to power the CEB press. However, the CEB press could benefit from faster gpm flow as the tractor we have is only 14 gpm compared to the rental units which can do 30 gpm. This would speed up brick making by about 2x. If we have many people out to help, this could significantly improve the efficiency of the operation, and may be worth the extra cost. On days when the CEB Press was not the bottleneck, Danny’s parents’ tractor could be used to power the CEB. The efficiency of construction with CEB blocks seems to be reliant on how to make sure that every step in chain from digging soil out of the ground to setting into the wall is happening at the same pace. If one step is slow, it will slow down all of the rest.

It would be nice to have a power source for the CEB in general, so we don’t always have to bring danny’s tractor down in order to use it. If someone else around town wanted to use the CEB press, then it would be much easier to loan them the press and power cube, but not danny’s tractor. They could also be instructed to rent machine X from the rental place … 5x 1 week rentals = cost of power cube

Build tolua++ files with makefile

Here is how you can have your makefile build your tolua++ .cpp and .h files for you. It should work for plain tolua also.

1
2
3
4
TOLUA = tolua++5.1

tolua_%.cpp tolua_%.h : %.pkg
$(TOLUA) -o $(@:%.h=%.cpp) -H $(@:%.cpp=%.h) $<

this will generate tolua_file.cpp and tolua_file.h files from corresponding file.pkg files anytime they the .cpp or .h file is depended on somewhere else in the file. In my case I just added tolua_file.o to my list of objects. Here is the full makefile for the project which required this - for reference:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
 

# LINUX
LIBLUA=lua5.1
# MAC OSX
#LIBLUA=lua

# LDFLAGS=-arch x86_64
OBJS = swarm.o group.o scene.o vmath.o tolua_group.o tolua_swarm.o tolua_vmath.o
CXX = g++
CXXFLAGS = -Wall -c -O2 `sdl-config --cflags`
LDFLAGS = -Wall `sdl-config --libs`
INCLUDES = -I./include -I/usr/include/lua5.1 -I/opt/local/include
LIBS = -L./lib -lANN -lGL -lGLU -llo -ltolua++5.1 -l$(LIBLUA)
TOLUA = tolua++5.1

tolua_%.cpp tolua_%.h : %.pkg
$(TOLUA) -o $(@:%.h=%.cpp) -H $(@:%.cpp=%.h) $<

%.o: %.cpp
$(CXX) $(INCLUDES) $(CXXFLAGS) -c $< -o $@

# the executable
swarm: $(OBJS)
$(CXX) $(LDFLAGS) -o $@ $^ $(LIBS)

download

Axpress

Query languages of the past have been good at locating data in existing databases. I wanted to see what it might look like to have a query language for not just local information but all available information and information yet to be generated. Here is what it looks like right now:

Examples

1
2
3

image[file.pattern] = "/home/dwiel/pictures/*.jpg"
image[file.filename] = _filename

This query returns a list of filenames of jpg images in my pictures directory:

1
2
3
4
5
6
7
8

[
{
'filename' : '/home/dwiel/pictures/sunset.jpg'
}, {
'filename' : '/home/dwiel/pictures/funny.jpg'
}
]

And another query:

1
2
3

image[file.pattern] = "/home/dwiel/pictures/*.jpg"
image[image.average_color] = _color

which this time just returns colors:

1
2
3
4
5
6
7
8

[
{
'color' : (244, 100, 133)
}, {
'color' : (24, 15, 12)
}
]

If we want both the filename and the color, we can request both

1
2
3
4

image[file.pattern] = "/home/dwiel/pictures/*.jpg"
image[file.filename] = _filename
image[image.average_color] = _color
1
2
3
4
5
6
7
8
9
10

[
{
'filename' : '/home/dwiel/pictures/sunset.jpg',
'color' : (244, 100, 133)
}, {
'filename' : '/home/dwiel/pictures/funny.jpg'
'color' : (24, 15, 12)
}
]

If we want to query for images on flickr it looks very similar (sorry about the lack of square brackets around flickr … wordpress tries to turn it into a flickr video) :

1
2
3

image flickr = "sunset"
image[image.average_color] = _color

Which atm, is (#2 is black and white):

1
2
3
4
5
6
7
8
9
10
11
12
13
14

[
{
'color' : [ 184, 139, 92, ],
}, {
'color' : 11,
}, {
'color' : [ 153, 117, 87, ],
}, {
'color' : [ 134, 138, 161, ],
}, {
'color' : [ 121, 119, 119, ],
},

]

Again, if we want a filename for these we can add that like we did before and the pictures will be downloaded to a temporary file as a side-effect so that there will be a filename to provide. Or we could just ask for a url.

For a more complex example, we’ll query for all of the artists similar to the artist that the last.fm user dwiel has most recently listened to:

1
2
3
4
5
6

user[lastfm.user_name] = 'dwiel'
user[lastfm.recent_track] = track
track[lastfm.artist] = artist
artist[lastfm.similar_to] = similar_artist
similar_artist[lastfm.artist_name] = _similar_name

or if you prefer, a more concise format:

1
2
3

user[lastfm.user_name] = 'dwiel'
user[lastfm.recent_track][lastfm.artist][lastfm.similar_to][lastfm.artist_name] = _similar_name
1
2
3
4
5
6
7
8
9
10

[
{
u'similar_name' : u'Scout Niblett',
}, {
u'similar_name' : u'Blonde Redhead',
}, ... {
u'similar_name' : u'The Fiery Furnaces',
},
]

Overall

As I explored this query language, one of the most interesting aspects was the ability to easily query for information which would otherwise require lots of glue code in other languages. If there is a translation from one data type to another, then you don’t need to code that translation explicitly. If you have an image URL or flickr tag, you can immediately start talking about it’s average color. This is very powerful, and I am still working out what kinds of new user interface designs that this may make possible.

One user interface design that I have been exploring would use this as an intermediate query language which some form of natural language could be compiled into. I am excited to explore where this goes when mixed with ideas from one of my previous projects english and some new ideas coming out of mozilla labs with ubiquity.

Code is available here. Though note that you must check out the latest svn as the version available under the download section is a very old version of the code. Back when it was merely a MQL style query language for accessing SPARQL endpoints. Although, if that is what you want, then the front page and the download files are applicable.

I have a simple web interface running locally, but I am not yet ready to let other people run arbitrary code on my machine (even in a sandbox). Hopefully soon.

Automatic Windows

Today I spent some time looking into automatic windows. Could windows be automatically be opened when the indoor and outdoor temperatures are at the right place? There are some windows we plan to put in the lofted area that will be too high to easily reach. These could also benefit from automatic window opening.

It seems that most solutions are pretty expensive. The cheapest/easiest solution seems to be using linear actuators. There are also window openers specifically made for this, but they tend to be even more expensive ($170) .

It seems that a lot of cost could be saved if regular DC motors could be fashioned to be used instead of pricey linear actuators. The actuators don’t get much cheaper than $110. DC motors would require more complex arrangement with chain or string and likely one for opening and one for closing the windows. These could probably be done, but would require quite a bit of experimentation first. It would cost significantly less, probably closer to $75 complete instead of $150 complete. These could be added later to double hung windows. One problem with these is that they can’t lock the window closed, also if the window is opened manually, the string will get messed up. Some kind of geared belt system might work better, but that gets way more complex …

One down side to the linear actuators is that due to the large cylinder, they might not be easy to fit in between a window and its screen. They make sense for casement windows and awning windows, but are more difficult for double hung windows. We have the benefit of 2’ deep walls, so it should be relatively easy to install a large 12” linear actuator in between our awning windows and their screen.

I also found these that seem to be reasonably priced and could fit in a much smaller space (it is 7 cm wide). They run on 24V. The actuator itself costs just around $100 + UK VAT? + shipping. They also come with lots of options for building a system with their accessories (power supply, remotes, etc.) One problem with these is that they are much less powerful than the general purpose linear actuators and only extend to 15cm (5.9 inches). You could mount them closer to the hinge to make the 5.9 inch opening act like more but I wouldn’t be confident that they would be strong enough to open the window with that kind of leverage.

One thing to consider before building, is that we might be able to run any necessary wiring through the walls, which I think would be preferable. Especially for above greenhouse windows. Another possibility is for double hung windows to have linear actuator built up into the wall, like in a lintel or something … The problem is that if we want the window to open 30 inches, there needs to be a 30 inch actuator inside …

Yet another possibility is making the linear actuators themselves out of a DC motor or servomotor. There are lots of designs, and they are really quite simple. The hardest part would be calculating the size of motor necessary to get the amount of force desired.

After all of this, I’ve decided that automatic windows are definitely not worth it at this point. I will still need to come up with a method for opening the windows that will be too high to open in a typical fashion, but will likely just use a simple manual pulley or push bar system. To be determined …