Changelog
[1.8.0] - 2023-06-26
This release contains the following service versions:
forge-sdk==5.1.0
(Python)forge-sdk==5.1.0
(Java)forge-cli==5.1.2
rule-engine==5.1.0
heartbeat==5.0.5
control-panel==0.0.6
cecs==5.0.3
Noteable changes since 1.7.0:
CLI
Fixed
- sometimes other services would accidentally pick up environment variables from the current service
Added
forge check
command which runs CECS checks on the project
CECS
Added
- significant performance improvements
Java SDK
Changed
- signal deserialization now uses class names to map the signals to correct classes (fixes
replyTo
)
Python SDK
Changed
- API builders send
messageType
inPascalCase
instead ofUPPER_SNAKE_CASE
- callbacks now support functions (in module root), and service methods can be passed in as static members or as strings
[1.7.0] - 2023-06-09
This release contains the following service versions:
forge-sdk==5.0.10
(Python)forge-sdk==5.0.11
(Java)forge-cli==5.1.0
rule-engine==5.0.12
heartbeat==5.0.5
control-panel==0.0.6
cecs==5.0.2
Noteable changes since 1.6.1:
CLI
Changed
https
istrue
by default now for exposed services (❗may be breaking)
Control Panel
Fixed
- YAML wasn't loading on startup
- terminal was not working
Rule Engine
Fixed
- sending huge messages would freeze the producer
NullPointerException
regardingsender
inAgentIdLogPattern
Agents.deleteAgent(agentId)
did not emit a data change signal
Added
Rule fire limit reached
exception prints which rules were fired
[1.6.1] - 2023-04-25
This release contains the following service versions:
forge-sdk==5.0.9
(Python)forge-sdk==5.0.10
(Java)forge-cli==5.0.10
rule-engine==5.0.10
heartbeat==5.0.4
control-panel==0.0.3
cecs==5.0.2
Noteable changes since 1.6.0:
CLI
Fixed
- hang after a failed build
Control Panel
Added
- terminal (alpha)
- send heartbeat action
Rule Engine
Added
Agents.deleteAgent(agentId)
Agents.getCreateOrDelete
Agents.getCreateOrDeleteByConnection
- utilities for agent tests (
assertRuleFired
,assertRuleNotFired
,getFactsOfType
, ...) Agent.removeConnection
Java SDK
Added
Utils.setFakeTime
for tests
[1.6.0] - 2023-04-11
This release contains the following service versions:
forge-sdk==5.0.8
(Python)forge-sdk==5.0.9
(Java)forge-cli==5.0.9
rule-engine==5.0.9
heartbeat==5.0.4
control-panel==0.0.1
cecs==5.0.2
Noteable changes since 1.5.5:
Control Panel
Added
- initial version (beta)
[1.5.5] - 2023-03-29
This release contains the following service versions:
forge-sdk==5.0.7
(Python)forge-sdk==5.0.9
(Java)forge-cli==5.0.8
rule-engine==5.0.9
heartbeat==5.0.4
cecs==5.0.2
Noteable changes since 1.5.4:
Rule Engine
Added
- unit testing agents
Fixed
- PostHog losing some events
CLI
Fixed
- bug where
forge
could sometimes only be run from the root of the project - correctly rebuilds Java services even when
autobuild
in VS Code is enabled - rebuilding API clients even for disabled services because of dependencies from other services
Added
- defining service limits in config
send-heartbeat
,delete-agent
andremove-pending-signals
now accept multiple agent IDs
Java SDK
Added
Utils.datetimeInTimezone
[1.5.4] - 2023-03-06
This release contains the following service versions:
forge-sdk==5.0.6
(Python)forge-sdk==5.0.6
(Java)forge-cli==5.0.7
rule-engine==5.0.6
heartbeat==5.0.4
cecs==5.0.2
Noteable changes since 1.5.3:
Java SDK
Fixed
- serializing
LocalTime
to MongoDB - faking time
- empty
Store
prefix
Changed
- replaced shutdown hooks with explicit
close()
calls
Added
callbackData
field to allMessage
signals
Python SDK
Fixed
- logger
replace
error when logging non-str objects
Changed
- replaced shutdown hooks with explicit
close()
calls
Added
callbackData
field to allMessage
signals
Rule Engine
Fixed
IllegalStateException
when shutting down
Added
RuleEngineAPI.removePendingSignals
to remove pending signals from an agent
CLI
Fixed
set-fake-now
,reset-fake-now
anddelete-topics
commands can now be run in sandbox environments
Added
forge run
andforge run-service
now check if there's a hanging service process and kill it before proceedingforge kill
to kill all hanging processesforge remove-pending-signals
to remove pending signals from an agent
Heartbeat
Fixed
- service freeze on database failure
- graceful shutdown on interrupt signal
Changed
- heartbeats ignore fake time to provide consistent experience when testing
Cecs
Fixed
- ignore markers in Java
[1.5.3] - 2023-02-02
This release contains the following service versions:
forge-sdk==5.0.4
(Python)forge-sdk==5.0.3
(Java)forge-cli==5.0.4
rule-engine==5.0.3
heartbeat==5.0.1
cecs==5.0.1
Noteable changes since 1.5.2:
CLI
Fixed
- build doesn't crash if there's an empty
clients/java/
folder - logs are streamed live into
forge.log
forge run
doesn't build excluded services
Changed
- config, dockerfiles and manifests are now rendered with the basic set of built-in values and filters (
project
,env
,os
,json
,read_file
,file_hash
,b64encode
andb64decode
)
Added
forge run
got the-ll/--log-level
option
Rule Engine
Fixed
UnsupportedOperationException
when there were pending signalsIllegalStateException
when shutting down- emitting agent data changes only when there were changes
Python SDK
Fixed
Module
now works correctly during migration toEnvironment
Java SDK
Changed
- updated
pulsar-client
dependency to2.11.0
- removed
netty
andaircompressor
version overrides
[1.5.2] - 2023-01-16
❗ This release contains important fixes. Please update as soon as possible.
This release contains the following service versions:
forge-sdk==5.0.3
(Python)forge-sdk==5.0.2
(Java)forge-cli==5.0.2
rule-engine==5.0.2
heartbeat==5.0.1
cecs==5.0.1
Noteable changes since 1.5.1:
Java SDK
Fixed
- reverted to using exact dependency versions since intervals caused build issues
Python SDK
Fixed
- Pulsar consumer now properly processes all pending messages and closes gracefully
- Pulsar producers are now flushed before closing
CLI
Fixed
consume-message
command- commands that send messages or events no longer need to sleep in order for messages to be flushed
Added
produce-message
now has--num-messages
option
[1.5.1] - 2023-01-11
This release contains the following service versions:
forge-sdk==5.0.2
(Python)forge-sdk==5.0.1
(Java)forge-cli==5.0.1
rule-engine==5.0.1
heartbeat==5.0.1
cecs==5.0.1
Noteable changes since 1.5.0:
CLI
Added
create-agent
command that makes it easier to create new agent typesnew-agent
,update-agent
,delete-agent
,send-heartbeat
commands that make it easier to manage agentssend-event
,send-message
andsend-agent-message
commands for manual interaction with the platformlist
andadd-service
commands that use the central service registry (experimental)
Changed
create-project
now accepts project name as an argumentSecrets
utility for changing the.env
file now handles comments correctly
Heartbeat
Changed
- deprecated
HeartbeatDisabled
feature toggle (useHeartbeatEnabled
instead)
Fixed
- warning log when the feature toggle was not set (now defaults to heartbeat enabled)
Rule Engine
Added
- logs now include the agent ID during agent evaluation
- when running locally, all agents are now unlocked when the rule engine starts
Salience
enum to allow for more readable salience values
Changed
- most dependencies are now intervals instead of exact versions
- agent IDs have 16 characters by default (instead of 8)
Removed
CutelogLayout
Python SDK
Fixed
- callbacks are now also
async
functions
Added
datetime.time
(de)serialization supporttime_to_str
andstr_to_time
utils
Changed
- deprecated
random_generator
(userandom_string
) - deprecated
get_datetime
,get_utc_datetime
,get_utc_date
,get_utc_datetime_str
andget_utc_date_str
(usenow
,today
,now_str
andtoday_str
) - deprecated
MongoClientKeeper
(useMongo
instead) - deprecated
Module
(useEnvironment
instead) - logs now escape the
\n
and\r
characters
Java SDK
Added
- (de)serialization support for Java 8
java.time
types (includingLocalTime
) andOptional
types strToTime
andtimeToStr
utils
Fixed
RejectedExecutionException
when shutting down the service- serializing only fields of a class and ignoring any additional
get*
methods
Changed
- most dependencies are now intervals instead of exact versions
- deprecated
Utils.randomGenerator
(useUtils.randomString
) - deprecated
getDatetime
,getUtcDatetime
,getUtcDate
,getUtcDatetimeStr
andgetUtcDateStr
(usenow
,today
,nowStr
andtodayStr
) - deprecated
MongoClientKeeper
(useMongo
instead) - deprecated
Module
(useEnvironment
instead) - logs now escape the
\n
and\r
characters
[1.5.0] - 2022-12-21
This release contains the following service versions:
forge-sdk==5.0.0
(Python)forge-sdk==5.0.0
(Java)forge-cli==5.0.0
rule-engine==5.0.0
heartbeat==5.0.0
cecs==5.0.0
Noteable changes since 1.5.0 beta:
CLI
Fixed
- loading env variables in JSON form
Added
- locking/unlocking all agents
- overriding any Kubernetes manifest file
Changed
- prebuilt Django services are started with
<service-name> runserver
instead of<service-name>-admin runserver
Rule Engine
Fixed
getByConnection
andgetOrCreateByConnection
now return the correct agent instance instead of a genericAgent
ConncurrentModificationExeption
when clearing entry points
Added
GetAgentByConnection
forsignals.yaml
- stopping agent evaluation with
Agents.stopEvaluation()
unlockAllAgents
andlockAllAgents
APIs
Changed
- deprecated
Agent.addConnection
(useAgent.setConnection
) - upgraded dependency
io.github.classgraph==4.8.151
Removed
json-simple
dependency
Python SDK
Fixed
- service API functions no longer get signal metadata (
id
,timestamp
...) in**kwargs
Changed
LOGGING
,REGISTRY_CONFIG
,CONSUMER_CONFIG
andPRODUCER_CONFIG
settings are now loaded as JSON to supported advanced configuration- upgraded dependencies (
redis<4.5.0
,fakeredis<3.0
)
Java SDK
Added
getVariant
inUnleashFeatureToggle
Changed
- requires self-hosted Sentry >=v21.9.0 (!)
- updated dependencies
Project template
Changed
- cleaned up project root
- using CI v5.x
Fixed
MONGO_CLIENT_CONNECTION_STRING
secret would throw an error if not setrule-engine
depends on themongo
database
Docs
Added
- technical documentation for CLI and Heartbeat
Fixed
- minor tutorial fixes
[1.5.0 beta] - 2022-10-19
This release contains the following service versions:
forge-sdk==5.0.0b0
(Python)forge-sdk==5.0.0b0
(Java)forge-cli==5.0.0b0
rule-engine==5.0.0b0
heartbeat==5.0.0b0
cecs==4.0.0a3
These are the most important changes when compared to v1.4. Aside from the ones mentioned, there were lots of bug fixes and improvements.
Concepts
Changed
Communication
- Instead of
BaseMessage
that hadconfiguration
andpayload
, there are now 5 clearly defined signal types:Message
- when agent/service wants to send a message to a specific agent/service. Triggers functions annotated with@api
Event
- when agent/service wants to notify its surroundings that something happened. Other agents/services can subscribe to a specific eventReply
- a subtype ofMessage
that can only be sent as a response to a previousMessage
signalErrorReply
- a subtype ofReply
that signalizes that the receivedMessage
couldn't be properly processedDataChangeEvent
- a special type of event that is sent when an "emittable" data model is created, updated or deleted. Other agents/services can subscribe to the changes of a specific data object
- All signals must contain the following fields:
id
- a unique string representing this signaltimestamp
- the datetime when this signal was createdsignalType
- one of:MESSAGE
,EVENT
,REPLY
,ERROR
,DATA_CHANGE
from
- sender's ID (service/agent ID)
- These signals are meant to be subclassed (additional fields can be added)
- Signals are serialized into a flat JSON structure
Message
- must define
messageType
which defaults to the name of the class (e.g.RegisterUser
) - used in deserialization - must define
to
- the recipient's ID - can define
returnAddress
to which the reply should be sent
- must define
Event
- must define
eventType
which must be unique in the whole system - defaults to the name of the class, used in deserialization - events no longer have a "service owner"
- must define
Reply
(extendsMessage
)- must define
replyTo
- theMessage
to which this is a response to, defaults to the message that's currently being processed - if no address is specified when sending, defaults to
replyTo.returnAddress
, and if that's not defined defaults to an address generated fromreplyTo.from
- must define
ErrorReply
(extendsReply
)- must define
code
anddescription
- can define
help
- must define
DataChangeEvent
- must define
changeType
- one of:CREATED
,UPDATED
,DELETED
- must define
objectType
- the type of object that has changed, defaults to the data model class name (e.g.User
) - used in deserialization - must define
object
- the new version of the object if created or updated, the last version of the object if deleted
- must define
- Instead of
Data models
- there is no longer a concept of separate data "model" and "view" because it created unnecessary complexity
- there is now only a single
DataModel
class which enabled the model to be saved to the database - if you want data object changes to be emitted automatically, inherit
EmittableDataModel
in Python, or set@DataModel(emit = true)
in Java
Added
Addresses
- Every service and agent now has an address on which it receives signals
- The service's default address is its name
- All agents have the same address (
agents
) - The signal has
from
andto
fields which define the specific recipient on that address (e.g. agent's ID) Message
signals can define areturnAddress
where they expect a reply
Agent locking
- when an agent is being evaluated, it is "locked" so that no other Rule Engine instances can start the evaluation of that agent
- this prevents concurrency problems and enables horizontal scaling of Rule Engine
- IMPORTANT: if an exception is thrown when an agent is being evaluated, it will stay locked and must manually be unlocked (either using CLI commands or the Rule Engine's API)
Concurrency
- All services (Python and Java) can now concurrently process multiple requests, thereby increasing performance (sometimes significantly)
- The level of concurrency is controlled with a
CONCURRENCY
setting (default: 1)
API Gateway
Changed
- renamed from "Forge API"
- changed
/signal/{from}/{to}/{type}
to/message/{to}
and/message/{to}/{type}
- changed
/event/{from}/{type}
to/event
and/event/{type}
CLI
Fixed
build
command would hang when build fails
Added
lock-agents
andunlock-agents
commands
Heartbeat
Changed
- moved to core services
- there are no longer two sub-services (
scheduler
andsender
) - there is now only a singleheartbeat
service SentHearbeat
view is now aHeartbeat
eventHeartbeat
view is now an emmitableHeartbeatModel
data model
Fixed
- handles deletion of agents correctly
Removed
- backoff strategy - can be implemented as a custom strategy
Rule Engine
Changed
send_signal
API is nowevaluate_with_signals
Evaluation
is now an event signal, usetimestamp
instead ofevaluatedAt
- project MUST include
kmodule.xml
Signals.on
is now eitherEvents.on
orDataChanges.on
- using
LocalDateTime
instead ofDate
- when an event is received, the Rule Engine determines which agents should receive it, and sends a message to each of them, instead of evaluating them all immediately and blocking other messages
- the agent's "summary" structure now has
facts
,pendingSignals
andpendingUpdates
getOrCreateByConnection
now automatically adds the connection when creating a new agent, so no need for the(connectionName, connectionId)
constructorgetOrCreate
now automatically sets the specified ID when creating a new agent
Added
- working with data changes is easier
AgentAlreadyExists
exception if trying to create an agent with an already existing IDGetOrCreateAgent
signal mapping strategy- custom signal mapping strategies can be created and used in the
yaml
configuration - locking agents when they're being evaluated
- any updates are queued as pending and applied after the evaluation
- if there are any queued signals, the agent's evaluation is queued again
- useful cron functions in
DateUtil
Removed
agent-created
entry-point - usesignals
- agents cannot send messages to specific entry points
- filters in subscriptions
- feature toggling - moved to SDK
SDK
Changed
- see also Concepts
Context
renamed toServiceContext
CurrentContext
is now a thread-localContext
which contains the signal that's currently being processed, and who's processing it (sender
)DBModel
is nowDataModel
- there are now 3 functions to build topic names:
get_input_topic_name
,get_event_topic_name
, andget_data_change_topic_name
forge.setup
now acceptsproject_slug
as an optional parameter
Added
- posthog integration
- support for a custom MongoDB connection string
@on_change
function decorator to listen to data changes- checks that an API is defined only once
utils.strings
- convert strings to camel/pascal/kebab/snake case
Removed
traceId
andpipeline
REGISTERED_VIEWS
setting - useservice.register_for_changes()
- constants
PROCESS
,CALLBACK
,EXCEPTION
DBView
- use data changes (EmittableDataModel
)EmitType
- useDataChangeType
build_configuration
and similar message building/parsing utils
Python SDK
Changed
- API functions now must be
async
- structure refactored (some imports may change)
- pydantic data-classes are now serialized by using the field's
alias
- bumped dependencies (
pydantic<1.11
,prometheus_client<0.16
)
Added
- some missing performance metrics on
Store
andCache
Store.delete
andCache.delete
Removed
forge.api.send_event
- just userequests
ServiceExtension
Java SDK
Added
Forge.setup
,ServiceContext
andContext
to mirror the Python SDKReflectionUtils
- ability to set a custom
Jinjava
template renderer - missing date-time and Base64 utilities to mirror Python SDK
- feature toggling utilities
DistributedCache
which equals to Python SDK'sCache(distributed=True)
Changed
- renamed
BaseSettings
toForgeSettings
- renamed
DataUtils
toDatabase
and added lots of useful functions @DataModel
should now only be used on data models that are saved in the database (not on signals)UnleashFeatureToggle
is now a singleton
Fixed
- methods whose name starts with
get
are no longer evaluated and serialized to MongoDB MONGO_REPLICA_SET_NAME
setting name