Александр Николаевич Власкин
Глоссариум по искусственному интеллекту: 2500 терминов. Том 2

«B»

Backpropagation through time (BPTT) is a gradient-based technique for training certain types of recurrent neural networks. It can be used to train Elman networks. The algorithm was independently derived by numerous researchers¹²⁷.

Backpropagation, also called «backward propagation of errors,» is an approach that is commonly used in the training process of the deep neural network to reduce errors¹²⁸.

Backward Chaining, also called goal-driven inference technique, is an inference approach that reasons backward from the goal to the conditions used to get the goal. Backward chaining inference is applied in many different fields, including game theory, automated theorem proving, and artificial intelligence¹²⁹.

Bag-of-words model in computer vision. In computer vision, the bag-of-words model (BoW model) can be applied to image classification, by treating image features as words. In document classification, a bag of words is a sparse vector of occurrence counts of words; that is, a sparse histogram over the vocabulary. In computer vision, a bag of visual words is a vector of occurrence counts of a vocabulary of local image features¹³⁰.

Bag-of-words model is a simplifying representation used in natural language processing and information retrieval (IR). In this model, a text (such as a sentence or a document) is represented as the bag (multiset) of its words, disregarding grammar and even word order but keeping multiplicity. The bag-of-words model has also been used for computer vision. The bag-of-words model is commonly used in methods of document classification where the (frequency of) occurrence of each word is used as a feature for training a classifier¹³¹.

Baldwin effect – the skills acquired by organisms during their life as a result of learning, after a certain number of generations, are recorded in the genome¹³².

Baseline is a model used as a reference point for comparing how well another model (typically, a more complex one) is performing. For example, a logistic regression model might serve as a good baseline for a deep model. For a particular problem, the baseline helps model developers quantify the minimal expected performance that a new model must achieve for the new model to be useful¹³³.

Batch – the set of examples used in one gradient update of model training¹³⁴.

Batch Normalization is a preprocessing step where the data are centered around zero, and often the standard deviation is set to unity¹³⁵.

Batch size – the number of examples in a batch. For example, the batch size of SGD is 1, while the batch size of a mini-batch is usually between 10 and 1000. Batch size is usually fixed during training and inference; however, TensorFlow does permit dynamic batch sizes¹³⁶,¹³⁷.

Bayes’s Theorem is a famous theorem used by statisticians to describe the probability of an event based on prior knowledge of conditions that might be related to an occurrence¹³⁸.

Bayesian classifier in machine learning is a family of simple probabilistic classifiers based on the use of the Bayes theorem and the «naive» assumption of the independence of the features of the objects being classified¹³⁹.

Bayesian Filter is a program using Bayesian logic. It is used to evaluate the header and content of email messages and determine whether or not it constitutes spam – unsolicited email or the electronic equivalent of hard copy bulk mail or junk mail. A Bayesian filter works with probabilities of specific words appearing in the header or content of an email. Certain words indicate a high probability that the email is spam, such as Viagra and refinance¹⁴⁰.

Bayesian Network, also called Bayes Network, belief network, or probabilistic directed acyclic graphical model, is a probabilistic graphical model (a statistical model) that represents a set of variables and their conditional dependencies via a directed acyclic graph¹⁴¹.

Bayesian optimization is a probabilistic regression model technique for optimizing computationally expensive objective functions by instead optimizing a surrogate that quantifies the uncertainty via a Bayesian learning technique. Since Bayesian optimization is itself very expensive, it is usually used to optimize expensive-to-evaluate tasks that have a small number of parameters, such as selecting hyperparameters¹⁴².

Bayesian programming is a formalism and a methodology for having a technique to specify probabilistic models and solve problems when less than the necessary information is available¹⁴³,¹⁴⁴.

Bees’ algorithm is a population-based search algorithm which was developed by Pham, Ghanbarzadeh and et al. in 2005. It mimics the food foraging behaviour of honey bee colonies. In its basic version the algorithm performs a kind of neighbourhood search combined with global search, and can be used for both combinatorial optimization and continuous optimization. The only condition for the application of the bee’s algorithm is that some measure of distance between the solutions is defined. The effectiveness and specific abilities of the bee’s algorithm have been proven in a number of studies¹⁴⁵.

Behavior informatics (BI) — the informatics of behaviors so as to obtain behavior intelligence and behavior insights¹⁴⁶.

Behavior tree (BT) is a mathematical model of plan execution used in computer science, robotics, control systems and video games. They describe switchings between a finite set of tasks in a modular fashion. Their strength comes from their ability to create very complex tasks composed of simple tasks, without worrying how the simple tasks are implemented. BTs present some similarities to hierarchical state machines with the key difference that the main building block of a behavior is a task rather than a state. Its ease of human understanding makes BTs less error-prone and very popular in the game developer community. BTs have shown to generalize several other control architectures¹⁴⁷.

Belief-desire-intention software model (BDI) is a software model developed for programming intelligent agents. Superficially characterized by the implementation of an agent’s beliefs, desires and intentions, it actually uses these concepts to solve a particular problem in agent programming. In essence, it provides a mechanism for separating the activity of selecting a plan (from a plan library or an external planner application) from the execution of currently active plans. Consequently, BDI agents are able to balance the time spent on deliberating about plans (choosing what to do) and executing those plans (doing it). A third activity, creating the plans in the first place (planning), is not within the scope of the model, and is left to the system designer and programmer¹⁴⁸.

Bellman equation – named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the «value» of a decision problem at a certain point in time in terms of the payoff from some initial choices and the «value» of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman’s «principle of optimality» prescribes¹⁴⁹.

Benchmark (also benchmark program, benchmarking program, benchmark test) – test program or package for evaluating (measuring and/or comparing) various aspects of the performance of a processor, individual devices, computer, system or a specific application, software; a benchmark that allows products from different manufacturers to be compared against each other or against some standard. For example, online benchmark – online benchmark; standard benchmark – standard benchmark; benchmark time comparison – comparison of benchmark execution times¹⁵⁰.

Benchmarking is a set of techniques that allow you to study the experience of competitors and implement best practices in your company¹⁵¹.

BETA refers to a phase in online service development in which the service is coming together functionality-wise but genuine user experiences are required before the service can be finished in a user-centered way. In online service development, the aim of the beta phase is to recognize both programming issues and usability-enhancing procedures. The beta phase is particularly often used in connection with online services and it can be either freely available (open beta) or restricted to a specific target group (closed beta)¹⁵².

Bias is a systematic trend that causes differences between results and facts. Error exists in the numbers of the data analysis process, including the source of the data, the estimate chosen, and how the data is analyzed. Error can seriously affect the results, for example, when studying people’s shopping habits. If the sample size is not large enough, the results may not reflect the buying habits of all people. That is, there may be discrepancies between survey results and actual results¹⁵³.

Biased algorithm – systematic and repetitive errors in a computer system that lead to unfair results, such as one privilege persecuting groups of users over others. Also, sexist and racist algorithms¹⁵⁴,¹⁵⁵.

Bidirectional (BiDi) is a term used to describe a system that evaluates the text that both precedes and follows a target section of text. In contrast, a unidirectional system only evaluates the text that precedes a target section of text¹⁵⁶.

Bidirectional Encoder Representations from Transformers (BERT) is a model architecture for text representation. A trained BERT model can act as part of a larger model for text classification or other ML tasks. BERT has the following characteristics: Uses the Transformer architecture, and therefore relies on self-attention. Uses the encoder part of the Transformer. The encoder’s job is to produce good text representations, rather than to perform a specific task like classification. Is bidirectional. Uses masking for unsupervised training¹⁵⁷,¹⁵⁸.

Bidirectional language model is a language model that determines the probability that a given token is present at a given location in an excerpt of text based on the preceding and following text¹⁵⁹.

Big data is a term for sets of digital data whose large size, rate of increase or complexity requires significant computing power for processing and special software tools for analysis and presentation in the form of human-perceptible results¹⁶⁰.

Big O notation is a mathematical notation that describes the limiting behavior of a function when the argument tends towards a particular value or infinity. It is a member of a family of notations invented by Paul Bachmann, Edmund Landau, and others, collectively called Bachmann—Landau notation or asymptotic notation¹⁶¹.

Bigram – an N-grams in which N=2¹⁶².

Binary choice regression model is a regression model in which the dependent variable is dichotomous or binary. Dependent variable can take only two values and mean, for example, belonging to a particular group¹⁶³.

Binary classification is a type of classification task that outputs one of two mutually exclusive classes. For example, a machine learning model that evaluates email messages and outputs either «spam» or «not spam» is a binary classifier¹⁶⁴.

Binary format is any file format in which information is encoded in some format other than a standard character-encoding scheme. A file written in binary format contains information that is not displayable as characters. Software capable of understanding the particular binary format method of encoding information must be used to interpret the information in a binary-formatted file. Binary formats are often used to store more information in less space than possible in a character format file. They can also be searched and analyzed more quickly by appropriate software. A file written in binary format could store the number «7» as a binary number (instead of as a character) in as little as 3 bits (i.e., 111), but would more typically use 4 bits (i.e., 0111). Binary formats are not normally portable, however. Software program files are written in binary format. Examples of numeric data files distributed in binary format include the IBM-binary versions of the Center for Research in Security Prices files and the U.S. Department of Commerce’s National Trade Data Bank on CD-ROM. The International Monetary Fund distributes International Financial Statistics in a mixed-character format and binary (packed-decimal) format. SAS and SPSS store their system files in binary format¹⁶⁵.

Binary number is a number written using binary notation which only uses zeros and ones. Example: Decimal number 7 in binary notation is: 111¹⁶⁶.

Binary tree is a tree data structure in which each node has at most two children, which are referred to as the left child and the right child. A recursive definition using just set theory notions is that a (non-empty) binary tree is a tuple (L, S, R), where L and R are binary trees or the empty set and S is a singleton set. Some authors allow the binary tree to be the empty set as well¹⁶⁷.

Binning is the process of combining charge from neighboring pixels in a CCD during readout. This process is performed prior to digitization in the CCD chip using dedicated serial and parallel register control. The two main benefits of binning are improved signal-to-noise ratio (SNR) and the ability to increase frame rates, albeit at the cost of reduced spatial resolution.

Bioconservatism (a portmanteau of biology and conservatism) is a stance of hesitancy and skepticism regarding radical technological advances, especially those that seek to modify or enhance the human condition. Bioconservatism is characterized by a belief that technological trends in today’s society risk compromising human dignity, and by opposition to movements and technologies including transhumanism, human genetic modification, «strong» artificial intelligence, and the technological singularity. Many bioconservatives also oppose the use of technologies such as life extension and preimplantation genetic screening¹⁶⁸,¹⁶⁹.

Biometrics is a people recognition system, one or more physical or behavioral traits¹⁷⁰,¹⁷¹.

Black box is a description of some deep learning system. They take an input and provide an output, but the calculations that occur in between are not easy for humans to interpret¹⁷²,¹⁷³.

Blackboard system is an artificial intelligence approach based on the blackboard architectural model, where a common knowledge base, the «blackboard», is iteratively updated by a diverse group of specialist knowledge sources, starting with a problem specification and ending with a solution. Each knowledge source updates the blackboard with a partial solution when its internal constraints match the blackboard state. In this way, the specialists work together to solve the problem¹⁷⁴.

BLEU (Bilingual Evaluation Understudy) is a text quality evaluation algorithm between 0.0 and 1.0, inclusive, indicating the quality of a translation between two human languages (for example, between English and Russian). A BLEU score of 1.0 indicates a perfect translation; a BLEU score of 0.0 indicates a terrible translation¹⁷⁵.

Blockchain is algorithms and protocols for decentralized storage and processing of transactions structured as a sequence of linked blocks without the possibility of their subsequent change¹⁷⁶.

Boltzmann machine (also stochastic Hopfield network with hidden units) is a type of stochastic recurrent neural network and Markov random field. Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield networks¹⁷⁷.

Boolean neural network is an artificial neural network approach which only consists of Boolean neurons (and, or, not). Such an approach reduces the use of memory space and computation time. It can be implemented to the programmable circuits such as FPGA (Field-Programmable Gate Array or Integrated circuit).

Boolean satisfiability problem (also propositional satisfiability problem; abbreviated SATISFIABILITY or SAT) is the problem of determining if there exists an interpretation that satisfies a given Boolean formula. In other words, it asks whether the variables of a given Boolean formula can be consistently replaced by the values TRUE or FALSE in such a way that the formula evaluates to TRUE. If this is the case, the formula is called satisfiable. On the other hand, if no such assignment exists, the function expressed by the formula is FALSE for all possible variable assignments and the formula is unsatisfiable¹⁷⁸.

Boosting is a Machine Learning ensemble meta-algorithm for primarily reducing bias and variance in supervised learning, and a family of Machine Learning algorithms that convert weak learners to strong ones¹⁷⁹.

Bounding Box commonly used in image or video tagging; this is an imaginary box drawn on visual information. The contents of the box are labeled to help a model recognize it as a distinct type of object.

Brain technology (also self-learning know-how system) is a technology that employs the latest findings in neuroscience. The term was first introduced by the Artificial Intelligence Laboratory in Zurich, Switzerland, in the context of the ROBOY project. Brain Technology can be employed in robots, know-how management systems and any other application with self-learning capabilities. In particular, Brain Technology applications allow the visualization of the underlying learning architecture often coined as «know-how maps»¹⁸⁰.

Brain—computer interface (BCI), sometimes called a brain—machine interface (BMI), is a direct communication pathway between the brain’s electrical activity and an external device, most commonly a computer or robotic limb. Research on brain—computer interface began in the 1970s by Jacques Vidal at the University of California, Los Angeles (UCLA) under a grant from the National Science Foundation, followed by a contract from DARPA. The Vidal’s 1973 paper marks the first appearance of the expression brain—computer interface in scientific literature¹⁸¹.

Brain-inspired computing – calculations on brain-like structures, brain-like calculations using the principles of the brain (see also neurocomputing, neuromorphic engineering).

Branching factor in computing, tree data structures, and game theory, the number of children at each node, the outdegree. If this value is not uniform, an average branching factor can be calculated¹⁸²,¹⁸³.

Broadband refers to various high-capacity transmission technologies that transmit data, voice, and video across long distances and at high speeds. Common mediums of transmission include coaxial cables, fiber optic cables, and radio waves¹⁸⁴.

Brute-force search (also exhaustive search or generate and test) is a very general problem-solving technique and algorithmic paradigm that consists of systematically enumerating all possible candidates for the solution and checking whether each candidate satisfies the problem’s statement¹⁸⁵.

Bucketing – converting a (usually continuous) feature into multiple binary features called buckets or bins, typically based on value range¹⁸⁶.

Byte – eight bits. A byte is simply a chunk of 8 ones and zeros. For example: 01000001 is a byte. A computer often works with groups of bits rather than individual bits and the smallest group of bits that a computer usually works with is a byte. A byte is equal to one column in a file written in character format¹⁸⁷.

«C»

CAFFE is short for Convolutional Architecture for Fast Feature Embedding which is an open-source deep learning framework de- veloped in Berkeley AI Research. It supports many different deep learning architectures and GPU-based acceleration computation kernels¹⁸⁸,¹⁸⁹.

Calibration layer is a post-prediction adjustment, typically to account for prediction bias. The adjusted predictions and probabilities should match the distribution of an observed set of labels¹⁹⁰.

Candidate generation — the initial set of recommendations chosen by a recommendation system¹⁹¹.

Candidate sampling is a training-time optimization in which a probability is calculated for all the positive labels, using, for example, softmax, but only for a random sample of negative labels. For example, if we have an example labeled beagle and dog candidate sampling computes the predicted probabilities and corresponding loss terms for the beagle and dog class outputs in addition to a random subset of the remaining classes (cat, lollipop, fence). The idea is that the negative classes can learn from less frequent negative reinforcement as long as positive classes always get proper positive reinforcement, and this is indeed observed empirically. The motivation for candidate sampling is a computational efficiency win from not computing predictions for all negatives¹⁹².

Canonical Formats in information technology, canonicalization is the process of making something conform] with some specification… and is in an approved format. Canonicalization may sometimes mean generating canonical data from noncanonical data. Canonical formats are widely supported and considered to be optimal for long-term preservation¹⁹³.

Capsule neural network (CapsNet) is a machine learning system that is a type of artificial neural network (ANN) that can be used to better model hierarchical relationships. The approach is an attempt to more closely mimic biological neural organization¹⁹⁴,¹⁹⁵.

Case-Based Reasoning (CBR) is a way to solve a new problem by using solutions to similar problems. It has been formalized to a process consisting of case retrieve, solution reuse, solution revise, and case retention¹⁹⁶.

Categorical data — features having a discrete set of possible values. For example, consider a categorical feature named house style, which has a discrete set of three possible values: Tudor, ranch, colonial. By representing house style as categorical data, the model can learn the separate impacts of Tudor, ranch, and colonial on house price. Sometimes, values in the discrete set are mutually exclusive, and only one value can be applied to a given example. For example, a car maker categorical feature would probably permit only a single value (Toyota) per example. Other times, more than one value may be applicable. A single car could be painted more than one different color, so a car color categorical feature would likely permit a single example to have multiple values (for example, red and white). Categorical features are sometimes called discrete features. Contrast with numerical data¹⁹⁷.

Center for Technological Competence is an organization that owns the results, tools for conducting fundamental research and platform solutions available to market participants to create applied solutions (products) on their basis. The Technology Competence Center can be a separate organization or be part of an application technology holding company¹⁹⁸.

Central Processing Unit (CPU) is a von Neumann cyclic processor designed to execute complex computer programs¹⁹⁹.

Centralized control is a process in which control signals are generated in a single control center and transmitted from it to numerous control objects²⁰⁰.

Centroid – the center of a cluster as determined by a k-means or k-median algorithm. For instance, if k is 3, then the k-means or k-median algorithm finds 3 centroids²⁰¹.

Centroid-based clustering is a category of clustering algorithms that organizes data into nonhierarchical clusters. k-means is the most widely used centroid-based clustering algorithm. Contrast with hierarchical clustering algorithms²⁰².

Character format is any file format in which information is encoded as characters using only a standard character-encoding scheme. A file written in «character format» contains only those bytes that are prescribed in the encoding scheme as corresponding to the characters in the scheme (e.g., alphabetic and numeric characters, punctuation marks, and spaces)²⁰³.

Сhatbot is a software application designed to simulate human conversation with users via text or speech. Also referred to as virtual agents, interactive agents, digital assistants, or conversational AI, chatbots are often integrated into applications, websites, or messaging platforms to provide support to users without the use of live human agents. Chatbots originally started out by offering users simple menus of choices, and then evolved to react to particular keywords. «But humans are very inventive in their use of language,» says Forrester’s McKeon-White. Someone looking for a password reset might say they’ve forgotten their access code, or are having problems getting into their account. «There are a lot of different ways to say the same thing,» he says. This is where AI comes in. Natural language processing is a subset of machine learning that enables a system to understand the meaning of written or even spoken language, even where there is a lot of variation in the phrasing. To succeed, a chatbot that relies on AI or machine learning needs first to be trained using a data set. In general, the bigger the training data set, and the narrower the domain, the more accurate and helpful a chatbot will be²⁰⁴.

Checkpoint — data that captures the state of the variables of a model at a particular time. Checkpoints enable exporting model weights, as well as performing training across multiple sessions. Checkpoints also enable training to continue past errors (for example, job preemption). Note that the graph itself is not included in a checkpoint²⁰⁵.

Chip is an electronic microcircuit of arbitrary complexity, made on a semiconductor substrate and placed in a non-separable case or without it, if included in the micro assembly²⁰⁶,²⁰⁷.

Class — one of a set of enumerated target values for a label. For example, in a binary classification model that detects spam, the two classes are spam and not spam. In a multi-class classification model that identifies dog breeds, the classes would be poodle, beagle, pug, and so on²⁰⁸.

Classification model is a type of machine learning model for distinguishing among two or more discrete classes. For example, a natural language processing classification model could determine whether an input sentence was in French, Spanish, or Italian²⁰⁹.

Classification threshold is a scalar-value criterion that is applied to a model’s predicted score in order to separate the positive class from the negative class. Used when mapping logistic regression results to binary classification²¹⁰.

Classification. Classification problems use an algorithm to accurately assign test data into specific categories, such as separating apples from oranges. Or, in the real world, supervised learning algorithms can be used to classify spam in a separate folder from your inbox. Linear classifiers, support vector machines, decision trees and random forest are all common types of classification algorithms²¹¹.

Сloud robotics is a field of robotics that attempts to invoke cloud technologies such as cloud computing, cloud storage, and other Internet technologies centred on the benefits of converged infrastructure and shared services for robotics. When connected to the cloud, robots can benefit from the powerful computation, storage, and communication resources of modern data center in the cloud, which can process and share information from various robots or agent (other machines, smart objects, humans, etc.). Humans can also delegate tasks to robots remotely through networks. Cloud computing technologies enable robot systems to be endowed with powerful capability whilst reducing costs through cloud technologies. Thus, it is possible to build lightweight, low cost, smarter robots have intelligent «brain» in the cloud. The «brain» consists of data center, knowledge base, task planners, deep learning, information processing, environment models, communication support, etc.²¹².

Clinical Decision Support (CDS) is a health information technology system that is designed to provide physicians and other health professionals with clinical decision support, that is, assistance with clinical decision- making tasks²¹³.

Clipping is a technique for handling outliers. Specifically, reducing feature values that are greater than a set maximum value down to that maximum value. Also, increasing feature values that are less than a specific minimum value up to that minimum value. For example, suppose that only a few feature values fall outside the range 40—60. In this case, you could do the following: Clip all values over 60 to be exactly 60. Clip all values under 40 to be exactly 40. In addition to bringing input values within a designated range, clipping can also used to force gradient values within a designated range during training²¹⁴.

Closed dictionary in speech recognition systems, a dictionary with a limited number of words, to which the recognition system is configured and which cannot be replenished by the user²¹⁵.

Cloud computing is an information technology model for providing ubiquitous and convenient access using the Internet to a common set of configurable computing resources («cloud»), data storage devices, applications and services that can be quickly provided and released from the load with minimal operating costs or with little or no involvement of the provider²¹⁶.

Cloud is a general metaphor that is used to refer to the Internet. Initially, the Internet was seen as a distributed network and then with the invention of the World Wide Web as a tangle of interlinked media. As the Internet continued to grow in both size and the range of activities it encompassed, it came to be known as «the cloud.» The use of the word cloud may be an attempt to capture both the size and nebulous nature of the Internet²¹⁷.

Cloud TPU is a specialized hardware accelerator designed to speed up machine learning workloads on Google Cloud Platform²¹⁸.

Cluster analysis is a type of unsupervised learning used for exploratory data analysis to find hidden patterns or groupings in the data; clusters are modeled with a similarity measure defined by metrics such as Euclidean or probability distance.

Clustering is a data mining technique for grouping unlabeled data based on their similarities or differences. For example, K-means clustering algorithms assign similar data points into groups, where the K value represents the size of the grouping and granularity. This technique is helpful for market segmentation, image compression, etc²¹⁹.