Improve documentation

README.md

@@ -48,9 +48,12 @@ In this library, an agent is defined as a side-effect function `ƒ: A ⟼ B`, wh
 - [Getting started](#getting-started)
 - [Quick example](#quick-example)
 - [Agent Architecture](#agent-architecture)
+  - [Memory](#memory)
+  - [Reasoner](#reasoner)
+  - [Encoder \& Decoder](#encoder--decoder)
   - [Commands \& Tools](#commands--tools)
-    - [Supported commands](#supported-commands)
-  - [Chaining agents](#chaining-agents)
+  - [Agent profiles](#agent-profiles)
+- [Agent chains](#agent-chains)
 - [FAQ](#faq)
 - [How To Contribute](#how-to-contribute)
   - [commit message](#commit-message)
@@ -66,7 +69,7 @@ Running the examples you need access either to AWS Bedrock or OpenAI.
 
 ## Quick example
 
-See ["Hello World"](./examples/helloworld/hw.go) application as the quick start. The example agent is `ƒ: string ⟼ string` that takes the sentence and returns the anagram. [HowTo](./doc/HOWTO.md) gives support to bootstrap it.
+See ["Hello World"](./examples/helloworld/hw.go) application as the quick start. The example agent is `ƒ: string ⟼ string` that takes the sentence and returns the anagram. [HowTo](./doc/HOWTO.md) gives support to bootstrap it. The library ships more [examples](./examples/) to demonstrate library's capabilities.
 
 ```go
 package main
@@ -77,13 +80,10 @@ import (
 
   // LLMs toolkit
   "github.com/kshard/chatter"
-  "github.com/kshard/chatter/bedrock"
+  "github.com/kshard/chatter/llm/autoconfig"
 
   // Agents toolkit
   "github.com/kshard/thinker/agent"
-  "github.com/kshard/thinker/codec"
-  "github.com/kshard/thinker/memory"
-  "github.com/kshard/thinker/reasoner"
 )
 
 // This function is core in the example. It takes input (the sentence)
@@ -116,26 +116,9 @@ func main() {
     panic(err)
   }
 
-  // We create an agent that takes string (sentence) and returns string (anagram).
-  agt := agent.NewAutomata(llm,
-    // Configures memory for the agent. Typically, memory retains all of
-    // the agent's observations. Here, we use a void memory, meaning no
-    // observations are retained.
-    memory.NewVoid(),
-
-    // Configures the reasoner, which determines the agent's next actions and prompts.
-    // Here, we use a void reasoner, meaning no reasoning is performed—the agent
-    // simply returns the result.
-    reasoner.NewVoid[string, string](),
-
-    // Configures the encoder to transform input of type A into a `chatter.Prompt`.
-    // Here, we use an encoder that converts string expressions into prompt.
-    codec.FromEncoder(anagram),
-
-    // Configure the decoder to transform output of LLM into type B.
-    // Here, we use the identity decoder that returns LLMs output as-is.
-    codec.DecoderID,
-  )
+	// Create an agent that takes string (sentence) and returns string (anagram).
+	// Stateless and memory less agent is used
+	agt := agent.NewPrompter(llm, anagram)
 
   // Evaluate expression and receive the result
   val, err := agt.Prompt(context.Background(), "a gentleman seating on horse")
@@ -175,13 +158,93 @@ graph TD
     end
 ```
 
+Following this architecture, the agent is assembled from building blocks as lego constructor:
+
+```go
+agent.NewAutomata(
+  // LLM used by the agent to solve the task
+  llm,
+
+  // Configures memory for the agent. Typically, memory retains all of
+  // the agent's observations. Here, we use a stream memory that holds all observations.
+  memory.NewStream(memory.INFINITE, "You are agent..."),
+
+  // Configures the reasoner, which determines the agent's next actions and prompts.
+  reasoner.From(deduct),
+
+  // Configures the encoder to transform input of type A into a `chatter.Prompt`.
+  // Here, we use an encoder that builds prompt.
+  codec.FromEncoder(encode),
+
+  // Configure the decoder to transform output of LLM into type B.
+  codec.FromDecoder(decode),
+)
+```
+
+The [rainbow example](./examples/rainbow/rainbow.go) demonstrates a simple agent that effectively utilizes the depicted agent architecture to solve a task.
+
+### Memory
+
 [`Memory`](./memory.go) is core element of agents behaviour. It is a database that maintains a comprehensive record of an agent’s experience. It recalls observations and builds the context windows to be used for prompting.
 
-[`Reasoner`](./reasoner.go) serves as the goal-setting component in the architecture. It evaluates the agent's current state, performing either deterministic or non-deterministic analysis of immediate results and past experiences. Based on this assessment, it determines whether the goal has been achieved and, if not, suggests the best new goal for the agent to pursue.
+The following [memory classes](https://pkg.go.dev/github.com/kshard/thinker/memory) are supported:
+* *Void* does not retain any observations.
+* *Stream* retains all of the agent's observations in the time ordered sequence. It is possible to re-call last N observations. 
+
+
+### Reasoner
+
+[`Reasoner`](./reasoner.go) serves as the goal-setting component in the architecture. It evaluates the agent's current state, performing either deterministic or non-deterministic analysis of immediate results and past experiences. Based on this assessment, it determines whether the goal has been achieved and, if not, suggests the best new goal for the agent to pursue. It maintain the following statemachine orchestrating the agent:
+
+```go
+const (
+	// Agent is asking for new facts from LLM
+	AGENT_ASK Phase = iota
+	// Agent has a final result to return
+	AGENT_RETURN
+	// Agent should retry with the same context
+	AGENT_RETRY
+	// Agent should refine the prompt based on feedback
+	AGENT_REFINE
+	// Agent aborts processing due to unrecoverable error
+	AGENT_ABORT
+)
+```
+
+The following [reasoner classes](https://pkg.go.dev/github.com/kshard/thinker/reasoner) are supported:
+* *Void* always sets a new goal to return results.
+* *Cmd* sets the goal for agent to execute a single command and return the result if/when successful.
+* *CmdSeq* sets the goal for reasoner to execute sequence of commands.
+* *From* is fundamental constuctor for application specific reasoners.
+* *Epoch* is pseudo reasoner, it limits number of itterations agent takes to solve a task.
+
+```go
+func deduct(state thinker.State[B]) (thinker.Phase, chatter.Prompt, error) {
+  // define reasoning strategy
+  return thinker.AGENT_RETURN, chatter.Prompt{}, nil 
+}
+
+reasoner.From(deduct)
+```
+
+### Encoder & Decoder
 
 The type-safe agent interface `ƒ: A ⟼ B` is well-suited for composition and agent chaining. However, encoding and decoding application-specific types must be abstracted. To facilitate this, the library provides two key traits: [`Encoder`](./codec.go) for constructing prompts and [`Decoder`](./codec.go) for parsing and validating LLM responses.
 
-The [rainbow example](./examples/rainbow/rainbow.go) demonstrates a simple agent that effectively utilizes the depicted agent architecture to solve a task.
+The following [codec classes](https://pkg.go.dev/github.com/kshard/thinker/reasoner) are supported:
+* *EncoderID* and *DecoderID* are identity codec taking and producing strings as-is.
+* *FromEncoder* and *FromDecoder* are fundamental constuctor for application specific codecs.
+
+```go
+// Encode type A to prompt
+func encoder[A any](A) (prompt chatter.Prompt, err error) { /* ... */ }
+
+// Decode LLMs response to `B` 
+func decoder[B any](reply chatter.Reply) (float64, B, error)  { /* ... */ }
+
+codec.FromEncoder(encoder)
+codec.FromDecoder(decoder)
+```
 
 ### Commands & Tools
 
@@ -191,16 +254,23 @@ When constructing a prompt, it is essential to include a section that "advertise
 
 The [script example](./examples/script/script.go) demonstrates a simple agent that utilizes `bash` to generate and modify files on the local filesystem.
 
-#### Supported commands
-* `bash` execute bash script or single command
-* `golang` execute golang code block
-* `python` execute python code block
+The following commands are supported
+* *bash* execute bash script or single command
+* *golang* execute golang code block
+* *python* execute python code block
+
+### Agent profiles
+
+The application assembles agents from three elements: memory, reasoner and codecs. To simplfy the development, there are few built-in profiles that configures it:
+* `Prompter` is ask-reply from LLM;
+* `Worker` uses LLMs and external tools to solve the task. 
+
 
-### Chaining agents
+## Agent chains
 
 The `thinker` library does not provide built-in mechanisms for chaining agents. Instead, it encourages the use of standard Go techniques either pure functional chaining or chaining of go routines (e.g. [golem/pipe](https://github.com/fogfish/golem)). 
 
-The [chain example](./examples/chain/chain.go) demostrates off-the-shelf techniques for agents chaining.
+The [chain example](./examples/05_chain/chain.go) demostrates off-the-shelf techniques for agents chaining.
 
 
 ## FAQ

agent/prompter.go

@@ -15,7 +15,7 @@ import (
 	"github.com/kshard/thinker/reasoner"
 )
 
-// Prompter is memoryless and stateless agent, implementing request/response to LLMs
+// Prompter is memoryless and stateless agent, implementing request/response to LLMs.
 type Prompter[A any] struct {
 	*Automata[A, string]
 }
@@ -24,9 +24,23 @@ func NewPrompter[A any](llm chatter.Chatter, f func(A) (chatter.Prompt, error))
 	w := &Prompter[A]{}
 	w.Automata = NewAutomata(
 		llm,
+
+		// Configures memory for the agent. Typically, memory retains all of
+		// the agent's observations. Here, we use a void memory, meaning no
+		// observations are retained.
 		memory.NewVoid(""),
+
+		// Configures the reasoner, which determines the agent's next actions and prompts.
+		// Here, we use a void reasoner, meaning no reasoning is performed—the agent
+		// simply returns the result.
 		reasoner.NewVoid[string](),
+
+		// Configures the encoder to transform input of type A into a `chatter.Prompt`.
+		// Here, we use an encoder that converts input into prompt.
 		codec.FromEncoder(f),
+
+		// Configure the decoder to transform output of LLM into type B.
+		// Here, we use the identity decoder that returns LLMs output as-is.
 		codec.DecoderID,
 	)
 

agent/worker.go

@@ -35,12 +35,26 @@ func NewWorker[A any](
 	w := &Worker[A]{encoder: encoder, registry: registry}
 	w.Automata = NewAutomata(
 		llm,
+
+		// Configures memory for the agent. Typically, memory retains all of
+		// the agent's observations. Here, we use an infinite stream memory,
+		// recalling all observations.
 		memory.NewStream(memory.INFINITE, `
 			You are automomous agent who uses tools to perform required tasks.
 			You are using and remember context from earlier chat history to execute the task.
 		`),
+
+		// Configures the reasoner, which determines the agent's next actions and prompts.
+		// Here, we use a sequence of command reasoner, it assumes that input prompt is
+		// the workflow based on command. LLM guided to execute entire workflow.
 		reasoner.NewEpoch(attempts, reasoner.NewCmdSeq()),
+
+		// Configures the encoder to transform input of type A into a `chatter.Prompt`.
+		// Here, it is defined by application
 		codec.FromEncoder(w.encode),
+
+		// Configure the decoder to transform output of LLM into type B.
+		// The registry knows how to interpret the LLM's reply and executed the command.
 		registry,
 	)
 

examples/helloworld/hw.go → examples/01_helloworld/hw.go

@@ -15,9 +15,6 @@ import (
 	"github.com/kshard/chatter"
 	"github.com/kshard/chatter/llm/autoconfig"
 	"github.com/kshard/thinker/agent"
-	"github.com/kshard/thinker/codec"
-	"github.com/kshard/thinker/memory"
-	"github.com/kshard/thinker/reasoner"
 )
 
 // This function is core in the example. It takes input (the sentence)
@@ -50,26 +47,9 @@ func main() {
 		panic(err)
 	}
 
-	// We create an agent that takes string (sentence) and returns string (anagram).
-	agt := agent.NewAutomata(llm,
-		// Configures memory for the agent. Typically, memory retains all of
-		// the agent's observations. Here, we use a void memory, meaning no
-		// observations are retained.
-		memory.NewVoid(""),
-
-		// Configures the reasoner, which determines the agent's next actions and prompts.
-		// Here, we use a void reasoner, meaning no reasoning is performed—the agent
-		// simply returns the result.
-		reasoner.NewVoid[string](),
-
-		// Configures the encoder to transform input of type A into a `chatter.Prompt`.
-		// Here, we use an encoder that converts string expressions into prompt.
-		codec.FromEncoder(anagram),
-
-		// Configure the decoder to transform output of LLM into type B.
-		// Here, we use the identity decoder that returns LLMs output as-is.
-		codec.DecoderID,
-	)
+	// Create an agent that takes string (sentence) and returns string (anagram).
+	// Stateless and memory less agent is used
+	agt := agent.NewPrompter(llm, anagram)
 
 	// Evaluate expression and receive the result
 	val, err := agt.Prompt(context.Background(), "a gentleman seating on horse")

examples/06_text_processor/processor.go

@@ -0,0 +1,100 @@
+//
+// Copyright (C) 2025 Dmitry Kolesnikov
+//
+// This file may be modified and distributed under the terms
+// of the MIT license.  See the LICENSE file for details.
+// https://github.com/kshard/thinker
+//
+
+package main
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/fogfish/golem/pipe/v2"
+	"github.com/fogfish/stream/lfs"
+	"github.com/kshard/chatter"
+	"github.com/kshard/chatter/llm/autoconfig"
+	"github.com/kshard/thinker/agent"
+	"github.com/kshard/thinker/codec"
+	"github.com/kshard/thinker/command"
+	"github.com/kshard/thinker/command/xfs"
+)
+
+func bootstrap(n int) (prompt chatter.Prompt, err error) {
+	prompt.WithTask(`
+		Use available tools to create %d files one by one with three or four lines of random
+		but meanigful text. Each file must contain unique content.`, n)
+
+	return
+}
+
+func processor(s string) (prompt chatter.Prompt, err error) {
+	prompt.WithTask(`Analyze document and extract keywords.`)
+
+	prompt.With(
+		chatter.Blob("Document", s),
+	)
+
+	return
+}
+
+func show(x string) string {
+	fmt.Printf("==> %s\n", x)
+	return x
+}
+
+func main() {
+	llm, err := autoconfig.New("thinker")
+	if err != nil {
+		panic(err)
+	}
+
+	// In this example, we need to mount two file systems, containing input and
+	// output data.
+	in, err := lfs.New("/tmp/script/txt")
+	if err != nil {
+		panic(err)
+	}
+	to, err := lfs.New("/tmp/script/kwd")
+	if err != nil {
+		panic(err)
+	}
+
+	// We need 10 files, let's use agents to get itls
+	fmt.Printf("==> creating files ...\n")
+	registry := command.NewRegistry()
+	registry.Register(command.Bash("MacOS", "/tmp/script/txt"))
+	init := agent.NewWorker(llm, 4, codec.FromEncoder(bootstrap), registry)
+	if _, err = init.Prompt(context.Background(), 13); err != nil {
+		panic(err)
+	}
+
+	// Creating the FileSystem I/O utility
+	rfs := xfs.New(in)
+	wfs := xfs.New(to)
+
+	// create worker to extract keywords from text files
+	wrk := agent.NewPrompter(llm, processor)
+
+	// Create processing pipeline
+	fmt.Printf("==> processing files ...\n")
+	ctx, cancel := context.WithCancel(context.Background())
+
+	// 1. Walk over file system
+	a, errA := rfs.Walk(ctx, "/", "")
+	// 2. Print file name
+	b := pipe.StdErr(pipe.Map(ctx, a, pipe.Pure(show)))
+	// 3. Read the file
+	c, errC := pipe.Map(ctx, b, pipe.Try(rfs.Read))
+	// 4. Process the file with agent
+	d, errD := pipe.Map(ctx, c, pipe.Try(xfs.Echo(wrk)))
+	// 5. Write agents output to the new file, preserving the name
+	e, errE := pipe.Map(ctx, d, pipe.Try(wfs.Create))
+	// 6. Remove input file
+	f, errF := pipe.Map(ctx, e, pipe.Try(rfs.Remove))
+
+	<-pipe.Void(ctx, pipe.StdErr(f, pipe.Join(ctx, errA, errC, errD, errE, errF)))
+	cancel()
+}