90% of the cache line is wasted or Instruction selection is only worth 10% of total performance

If data layout is the main contributor in application performance, why isn’t there any tools to help program data layout?

Code shouldn’t have such drastically different cache performance, code we have written that works should have maximum (reliable) performance regardless.

I’m thinking of automatic batching and automatic layout algorithms to place data in a way that is efficient for the code in question.

So you could write some code and have the layout transformed by the data layout compiler.

The layout and the code that accesses the data must be rewritten to accommodate the different styles of accessing batched data. The actual layout and code executed should be transparent to the programmer. It would make debugging harder but would improve performance.

Array of Structures and Structures of Array should be hidden from the programmer with code written to be ergonomic as possible. The transformation should happen behind the scenes.

Hyperwork – How making money should be socialized

Why is it so hard to make money? There’s never quite enough of it.

You should be able to make money by logging into a website and doing tasks listed. It should be simple, fun, easy. You should have the money in your bank account by the end of the day.

The secrets of making money are fiercely guarded. Society always has some problems that need solving, why not share the problems that need solving (the demand), by attempting to give them prices and have a marketplace of problems.

For example, my library needs a lick of paint. I should be able to vote for this, provide a stake how much I’m willing to pay towards a lick of paint and then people should be able to bid on the work. Get paid if they paint the library.

I should be able to list something I want and the price I’m willing to pay for it. I should be able to see a list of the tasks that need doing and prices that people are willing to pay for it.

The Struggle to Stay Relevant

You need to understand the economy (and some economics) to do well in society. This stuff isn’t taught at school or universities. Our manufacturing economies have been gutted by outsourcing and restructuring. You have to be aggressively creating value to succeed in our economy which has been gutted. In other words, you have to be prepared to work hard. doing things that are unusual. What if there is an intelligent way to be paid well while working at an average level?

It doesn’t look good. Wage stagnation is the long term trend. Labour has lost the war against capital. You cannot rest your laurels. So I recommend you aggressively save money. Eventually if you save enough, you won’t have to work.

The answer is often just learn to code. That’s what everyone else is doing. An army of low quality programmers awaits to do your React app. Pushing the prices of software engineering even lower than today. I find this solution to be wholly inadequate and unsatisfying.

Software engineering will be gutted too as it gets outsourced. Covid has shown that work from home is more productive and efficient than working in a noisy open plan office.

Ideas for computers in the 21st Century

I think about creating a manual for a theoretical computer system that would be detailed, with rendered screenshots and act as a guide on how to implement such a theoretical system.

Some features that are missing in computers

Instant sign up

The computer should begin collecting information about you, for you. It should collect your first name, surname, email address so you can instantly sign up to any online services. This is the instant sign up feature.

Payment integrations

The computer should be integrated with world markets and payment systems. Not as an app but as a native feature. It should be possible to collect payments with just a desktop computer, with a simple point of sale interface.

Distributed by default

A desktop computer should attempt to form connections with other computers for performance. The desktop computer should support decentralized and decentralized operation. For traditional systems this is called scaling out. A desktop operating system should scale out. Searches, computations, running programmes should run atop multiple computers for performance reasons.

So I should be able to connect my laptop and desktop into a cluster and have them both contribute computation and storage.

Single file system

We want a file system that abstracts across storage media in a way that makes sense to normal people. So you can have one giant file system with all your files composed from lots of smaller file systems such as Synalogy RAID drives, internal hard drives etc. GlusterFS does this but is rather advanced software to use and configure. As simple as AWS Elastic File System to use but composed from the user’s disks and drives.

Spreadsheet pipelines

How do we create communication protocols not in code?

It would be nice if we could create layers of behaviour with spreadsheet cells that are not mathematical but are communication. I’d like to be able to create an audio stream in spreadsheet and delineate a cell that is on another computer and have that audio go to that computer and create a cell outputting to the current computer coming from the other computer, this would be a basic audio chat.

Creating a devops pipeline should be something that anybody can do with a few minutes time. So I propose a spreadsheet like interface for creating pipelines.

International work ticketing system

The ability to request help with some work on a computer. Define a piece of work, inputs, expected outputs and raise the ticket on a work ticketing system that is public that anybody can use. Would be used by the desktop environment to make new features available and to encourage users to develop features together.

Additive GUIs thinking

Additive UIs use a different serialization to standard UI technology which uses trees. They use triples. Triples form a graph and from the resulting graph you can formulate trees and other complex data structures.

Are you tired of WYSIWYG and drag and drop? Then additive GUIs may be the alternative you’re looking for. If you’re tired of doing layouts by hand, you may see merit in the additive GUI approach to building user interfaces.

Building editors

Additive GUIs lend themselves to creating editors because of self-introspection. User updatable trees are easy to create. Creating a dynamic view that renders itself is simple.

For example, to create a simple editor of trees, which acts about like a todo list, a bit like todo mvc:

var template = {
    "predicates": [
		"NewTodo leftOf insertButton",
		"Todos below insertButton",
                "Todos backedBy todos",
                "Todos mappedTo todos",
                "Todos key .description",
		"Todos editable $item.description",
		"insertButton on:click insert-new-item",
		"insert-new-item 0.pushes {\"description\": \"$item.NewTodo.description\"}",
		"insert-new-item 0.pushTo $item.todos",
		"NewTodo backedBy NewTodo",
		"NewTodo mappedTo editBox",
		"NewTodo editable $item.description",
		"NewTodo key .description"
    ],
    "widgets": {
        "todos": {
            "predicates": [
                "label hasContent .description"
            ]
        },
	"editBox": {
	     "predicates": [
	        "NewItemField hasContent .description"
	     }	
	}
    },
    "data": {
	"NewTodo": {
	     "description": "Hello world"
	},
        "todos": [
            {
                "description": "todo one"
            },
            {
                "description": "todo two"
            },
            {
                "description": "todo three"
            }
        ]
    }
}

Additive GUIs do the same thing as JSX. They create a render tree of React.createElement that is similar to how JSX is compiled to. Except you don’t have to design the exact render tree yourself, you simply put together a tree of React components into a JSON document.

Design language

Additive GUIs provide a design language that describes how GUIs are composed. From “screens”, to “widgets”, to “dialogs”.

How additive GUIs work

Additive GUIs is a GUI technology whereby GUIs are constructed not from source code but from a DSL that makes statements about the output GUI. Additive GUIs should be useable to create complex GUIs such as mail clients and IDEs. By making statements about the output interface rather than specifying the render tree, we can describe what we want and obtain what you say is what you get.

By solving most UI objectives as predicates, the framework can abstract many common UI objectives away from the user without requiring custom code. Things like switching between widgets/screens/dialogs at run time have a canonical solution.

Additive GUIs are powered by trees that are managed for you. By creating a tree behind the scenes, additive GUIs can handle any traditional GUI. With additive GUIs you define the dataflow of your data.

In a traditional GUI we are managing a tree of items, which is traditionally hard to do and requires dedicated software to manage, such as the GUI builder within Android Studio. Instead of manually managing complex trees, the Additive GUI runtime manages the tree for you.

By focusing primarily on the output of the GUI, we can make direct adjustments to what we want the layout to be. We invent GUIs that can be modified far more easily at run time. Another property of additive GUIs that is useful is that the instructions that build up the GUI do not need to arrive all at once. A human can apply additive GUI statements piecemeal.

Additive GUIs use a runtime that runs in the background rendering the GUI for you. This runtime is capable of handling the different rules of the GUI seamlessly.

Modifying the tree is as simple as defining more propositional statements about widgets on the screen, providing a descriptive qualifier. The actual render tree is somewhat hidden from the author in an additive GUI but is generated by the structure of the additive GUI element definitions.

Advanced features

Additive GUIs have another trick up their sleeve. They can be extended at runtime to provide functionality that the authors did not consider. An example is a data grid that displays information about data items. Someone can right click this grid and click Insert rightOf and describe a new item on the GUI using additive GUI statements. People will be able to customize their desktops with custom widgets a bit like an Excel spreadsheet. With see it use it, users can make references to other things on the screen and convert existing data into different representations.

I hope that this feature provides an alternative to the OLE style of embedded GUIs which is very complicated and a specialist skill to implement. An Additive GUI can be changed by anybody and updating additive GUIs is a simple operation that is used with regularity like copy and paste.

Additive GUIs must provide the ability to switch between different trees, for example, in Chrome Developer tools there is a feature that allows the controlling of where the developer tools appears, docked to the left, bottom or right of the screen. This functionality should be easy to provide with an additive GUI.

The ability to create IDEs and mail clients with additive GUIs is a key feature, as IDEs are typically the most complex GUIs.

In an additive GUI, the programmer defines the relationships of data, or the model and what should appear on the screen. This is a mapping operation. The data is mapped to create widgets. The data flows through the GUI by the user interacting with the GUI.

Predicate join – remixing GUIs at runtime

Behind the scenes, data is being looped over, through a map operation. To map collections of items into widgets for display. Sometimes we might want to take data from two or more collections and join it together at display time. This is especially useful when customizing a running GUI. We might want to enrich the GUI with some additional data with a custom widget if an item is in a list.

To return to the todo example: we have two lists of data, completed todos and todo descriptions. We want to map them each into a widget and then join those widgets together, regardless of where they appear on the screen. We can make brand new items appear near where the data is currently being used or in completely unique places.

Here’s how we might join those two data sources together by placing the todo tick to the right of description.

{
	"predicates": [
		"todos mappedBy todoItem",
		"todos backedBy todos"
		"completeds backedBy completeds",
		"completeds mappedBy completeds",
		"todos joinField .id",
		"completeds joinField .id",
		"todos,completeds join joined"
	],
	"widgets": {
		"completeds": {
			"predicates": [
				"tick showIf .completed"
			]
		},
		"todoItem": {
			"predicates": [
				"description leftOf id",
				"id hasContent .id",
				"description hasContent .description"
			]
		},
		"joined": {
			"predicates": [
				"tick rightOf description"
			]
		}
	},
	"data": {
		"completeds": [
			{ "id": 1, "completed": false },
			{ "id": 2, "completed": true },
			{ "id": 3, "completed": false }
		],
		"todos": [
			{ "id": 1, "description": "work item 1" },
			{ "id": 2, "description": "work item 2" }
			{ "id": 3, "description": "work item 3" }
		]
	},
	
}

Business logic

GUIs are cookie cutter in additive GUIs. They should behave roughly the same way because they’re implemented with the same primitives. So you should get consistency for free. But you still have complete freedom over how the UI renders.

How to implement custom logic in an additive GUI?

If we use react as the boundary of a component, we lose the flexibility of the layout primitives.

Standard designs

Buy freedom with cash

Should you quit your job to do what you want? It’s a bad idea. But is it really what you need?

A break from corporate life. Peace. Freedom. Being your own boss. I hate being micromanaged at work so I feel like quitting. But I also feel I deserve a holiday in the prime of my life. I am almost 31 years old, I am tired of alarm clocks. I am tired of nit picky code reviews.

I have enough cash to ride it out for a couple of years then I will have to work again.

A blog in so many lines of code

The following:

class BlogTitle extends React.Component {
	constructor(props) {
		super(props);
	}
	render() {
		return <h2>{this.props.title}</h2>;
	}
	
}

class Title extends React.Component {
	constructor(props) {
		super(props);
	}
	render() {
		return <h1>Sam's Blog</h1>;
	}
	
}
 
// Sam Squire
var template = {
	"data": {
		"posts": [
			{
				"title": "Additive GUI blog",
				"body": "It's great to use additive GUIs to blog."
			},
				{
				"title": "Additive GUI blog",
				"body": "It's great to use additive GUIs to blog."
			}
		]
	},
	"predicates": [
		"header above content",
		"header hasClass header"
	
	],
	"widgets": {
		"header": {
			"predicates": [
				"logo leftOf search"
			]
		},
		"content": {
			"predicates": [
				"title",
				"blogs",
				"title above blogs",
				"blogs backedBy .posts",
				"blogs mappedTo blogItem",
				"blogs key .title",
				"blogs hasClass offset-md-2"
			]
		},
		"blogTitle": {"react": BlogTitle},
		"title": {"react": Title},
		"blogItem": {
			"predicates": [
				"blogTitle above blogBody",
				"blogBody hasContent .body",
			
			],
		}
	}
};

Renders to this.

See additive-guis for more.

The world is awful

Suicide rates are going up for men. People are losing their livelihood. Houses are unaffordable for normal people. China is sending Muslims to concentration camps. There is a war on cash.

Everything is set to get that much worse, there’s a lack of compassion from the world’s elites. Poverty hasn’t changed.

Elites are quite simply not doing anything that advances the interests of the poor, sick and disabled. They only work toward things that advance the interests of the wealthy and powerful. So this group of people is shafted by laws and regulations.

Regulation is suffocating entire industries from competing. So most industries have sizable moats around their businesses that keeps competition out. Anti-trust law seems to sleep for years at a time while industries do what they want.

Software is a mess. Software suffers from unnecessary complexity and slowness. Latency is awful.

Labour has lost the capital versus labour war.

I don’t really know what to make of all this. But it’s pretty dire on the front lines.

Creating a database

You can create a fast key-value store database with a trie and a hashmap.

class Tree():
    def __init__(self, value, partition_key, lookup_key):
        self.value = value
        self.partition_key = partition_key
        self.lookup_key = lookup_key
        self.left = None
        self.right = None

    def insert(self, value, partition_key, lookup_key):
        if self.left == None and value <= self.value:
            self.left = Tree(value, partition_key, lookup_key)
        elif self.right == None and value > self.value:
            self.right = Tree(value, partition_key, lookup_key)
        elif value > self.value:
            self.right.insert(value, partition_key, lookup_key)
        elif value < self.value:
            self.left.insert(value, partition_key, lookup_key)
        elif self.value == "":
            self.value = value
            self.partition_key = partition_key
            self.lookup_key = lookup_key
        return self

    def walk(self, less_than, stop):
        if self.left:
            yield from self.left.walk(less_than, stop)
        if less_than <= self.value and self.value <= stop:
            yield self.partition_key, self.value, self.lookup_key
        if self.right:
            yield from self.right.walk(less_than, stop)

class Db():
    def __init__(self):
        self.db = {}
        self.sort_index = pygtrie.CharTrie()
        self.between_index = {}

    def store(self, partition_key, sort_key, item):
        lookup_key = partition_key + ":" + sort_key
        self.db[lookup_key] = item
        self.sort_index[lookup_key] = sort_key
        if sort_key not in self.sort_index:
            self.sort_index[sort_key] = pygtrie.CharTrie()
        self.sort_index[sort_key][partition_key] = lookup_key
        if partition_key not in self.between_index:
            self.between_index[partition_key] = Tree("", None, None)
        self.between_index[partition_key].insert(sort_key, partition_key, lookup_key)

    def query_begins(self, partition_key, query, sortmode):
        def items():
            for sort_key, lookup_key in self.sort_index.iteritems(prefix=partition_key + ":" + query):
                yield (sort_key, lookup_key, self.db[sort_key])
        return sorted(items(), key=lambda x: x[0], reverse=sortmode == "desc")


    def query_pk_begins(self, partition_key, query, sortmode):
        def items(partition_key, query):
            for sort_key, value in self.sort_index.iteritems(prefix=partition_key):
                for partition_key, value in value.iteritems(prefix=query):
                    yield (sort_key, partition_key, self.db[value])
        return sorted(items(partition_key, query), key=lambda x: x[0], reverse=sortmode == "desc")

    def query_between(self, partition_key, from_query, to_query, sortmode):
        def items(partition_key, from_query, to_query):
            for partition_key, sort_key, lookup_key in self.between_index[partition_key].walk(from_query, to_query):
                yield (sort_key, partition_key, self.db[lookup_key])
        return sorted(items(partition_key, from_query, to_query), key=lambda x: x[0], reverse=sortmode == "desc")

    def query_before_than(self, partition_key, prefix, target_sort_key, sortmode):
        def less_than():
            for lookup_key, sort_key in self.sort_index.items(prefix=partition_key + ":" + prefix):
                if sort_key < target_sort_key:
                    yield (sort_key, lookup_key, self.db[lookup_key])
        return sorted(less_than(), key=lambda x: x[0], reverse=sortmode == "desc")

    def query_greater_than(self, partition_key, prefix, target_sort_key, sortmode):
        def less_than():
            for lookup_key, sort_key in self.sort_index.items(prefix=partition_key + ":" + prefix):
                if sort_key > target_sort_key:
                    yield (sort_key, lookup_key, self.db[lookup_key])
        return sorted(less_than(), key=lambda x: x[0], reverse=sortmode == "desc")

This sourcecode allows range queries too!


db = Db()
db.store("user#samsquire", "following#dinar", ["Messages 1"])
db.store("user#samsquire", "following#someonelse", ["Messages 2"])
db.store("user#samsquire", "message#2020-05-01T14:39", ["Messages 1"])
db.store("user#samsquire", "profile", ["profile"])
db.store("user#samsquire", "message#2020-06-01T14:39", ["Messages 2"])
db.store("user#samsquire", "message#2020-07-01T09:30", ["Messages 3"])
db.store("user#samsquire", "message#2020-08-01T14:39", ["Messages 4"])
db.store("user#dinar", "message#2020-09-01T14:39", ["Messages 5"])

print("find followers of user user#samsquire")
print(list(db.query_begins("user#samsquire", "following", sortmode="asc")))

print("find messages by samsquire between dates")
print(list(db.query_between("user#samsquire", "message#2020-06-01", "message#2020-07-01~~~~~~~~~", sortmode="asc")))

print("find all users who sent messages")
print(list(db.query_pk_begins("message", "user", sortmode="desc")))

print("before than")
print(list(db.query_before_than("user#samsquire", "message", "message#2020-07", "asc")))

print("greater than")
print(list(db.query_greater_than("user#samsquire", "message", "message#2020-07", "asc")))