Beating Transformers Using Synthetic Cognition

Alfredo Ibias

doi:10.1007/978-3-032-00686-8_26

Beating Transformers Using Synthetic Cognition

2026-01-01

0

PRE

AI

A

Alfredo Ibias *

M

Miguel Rodriguez-Galindo

H

Hector Antona

G

Guillem Ramirez-Miranda

E

Enric Guinovart

DOI：10.1007/978-3-032-00686-8_26

Original

Original request for help

Save

Abstract

En 中文

The road to Artificial General Intelligence goes through the generation of context-aware reactive behaviors, where the Transformer architecture has been proven to be the state-of-the-art. However, they still fail to develop reasoning. Recently, a novel approach for developing cognitive architectures, called Synthetic Cognition, has been proposed and implemented to develop instantaneous reactive behavior. In this study, we aim to explore the use of Synthetic Cognition to develop context-aware reactive behaviors. We propose a mechanism to deal with sequences for the recent implementation of Synthetic Cognition, and test it against DNA foundation models in DNA sequence classification tasks. In our experiments, our proposal clearly outperforms the DNA foundation models, obtaining the best score on more benchmark tasks than the alternatives. Thus, we achieve two goals: expanding Synthetic Cognition to deal with sequences, and beating the Transformer architecture for sequence classification.

Keywords:

Sequence Classification

Primitive-based Models

Transformers

Beating Transformers Using Synthetic Cognition

Abstract

Journal

Organization