Как стать автором
Обновить

Collective meaning recognition

Поисковые технологии *Семантика *Алгоритмы *Natural Language Processing *
Перевод
Автор оригинала: Sergey Pshenichnikov

The published material is in the Appendix of my book [1]

Modern civilization finds itself at a crossroads in which to choose the meaning of life. Because of the development of technology, the majority of the world's population may be "superfluous" - not in demand in the production of values. There is another option, where each person is a supreme value, an absolute individual and can be indispensably useful in the technology of the collective mind.

In the eighties of the last century, the task of creating a scientific field of "collective intelligence" was set. Collective intelligence is defined as the ability of the collective to find solutions to problems more effectively than each participant individually. The right collective mind must be superior in every measure to the mind of any member of the collective.

The problem with creating the right collective mind is the need for mutual understanding within the team. When there are more than five specialists, they may not understand a single term in the same way [2]. One's knowledge depends on one's phenotype-education, academic and elementary school, and personal experience. In this respect, the group has a greater chance of turning into a crowd, when the ability of the collective is reduced to the capabilities of its most "simple" participant. At this primitive level, mutual understanding in a crowd is achieved by analogy with the steady state in mechanics.

The collective mind is not only people, but also the tools-technologies to support their mental abilities, which are derived from natural and special languages. Human mental abilities are a highly individual phenomenon, so each person's language is absolutely personal. The comparison of people's linguistic contexts is the basis of the technology of understanding.

Ignoring the existence of multiple meanings of sign sequences has always led to the collapse of hopes and the death of people, communities and even civilizations. The history of world civilizations, in particular, is a description of the triggers of misunderstanding.

1. Historical Examples

1.1 The Persians and the Scythians

It is impossible to name the time and place of the creation of writing and mathematics. It seems to be ancient Egypt, but the texts and formulas were depicted on papyri, which were not preserved. The period of Hammurabi's reign in Babylon is referred to in terms of the preservation of the medium because of the sign sequences depicted by the Babylonians on clay tablets.

It seems attractive for the theme of collective intelligence the version by Herodotus that the authorship of the first meaningful sign sequence belongs to the Scythians, who gave to Darius 1 bird, mouse, frog and five arrows. According to Solomonik's classification [3], the Scythians used a natural sign system in which the signs were real objects rather than symbols.

The preference of Herodotus' version about the authorship is that there are at least two meanings in the message of the Scythians. The possibility of multiple interpretations is a defining property of such sign sequences. The semantic plurality of texts was immediately apparent in this text-message. Also important is the first experience of collective recognition of the meaning of the Scythian message.

The main thing the Persians did immediately was to assemble a council of experts to collectively search for the meaning of the text-message. Darius, naturally as chairman of the council of experts, formulated his version: the Scythians surrendered themselves to his power and so brought him land and water as a sign of submission, since the mouse lives in the land, the frog lives in the water, the bird, fast as a horse, is a sign of flight, and the arrows mean that the Scythians refuse to resist. A non-royal expert version has also been suggested: unless you Persians, like birds, fly into the sky, or, like mice, burrow into the ground, or, like frogs, leap into the marsh, you will not come back smitten by these arrows.

The Persian brainstorming resulted in two opposing interpretations of the same iconic sequence.

In this case, the first historical attempt at collective recognition of meaning must be considered a failure, although it is believed that the sage advisers were right. According to the algebraic approach to the analysis of sign sequences, the Persian expert council should have considered the contexts (hyperbinary phantom multipliers) of all sent signs, the context of that historical moment and calculated the overall context by means of reductions and Neuter chains.

In those days, the sign of submission to the conqueror was the sign sequence "land and water" as an ultimatum, presented by the Persian authorities to all cities, territories and consisted in demanding total submission. But if Darius is right, the Scythians would have sent a sack of sand and water without hesitation. If the consultants are right, the purpose of the message and the meaning of the threats are unclear. The Scythians were disproportionately weaker than the Persian army militarily, but not mentally.

The Scythians nevertheless sent this set. Why? That was the question Darius should have asked his experts, if he had not been an arrogant fool. But it was precisely to show Darius as an idiot that was the purpose of the Scythians, who sent Darius a detailed plan of their future actions, consisting of eight signs of their strategy and tactics. To destroy the reputation of King Darius as a great warrior.

The key sign of the Scythian military secret presented a priori to the Persians was the amphibious frog (context - earth and water). It was she who led Darius and his wise men to ordinary but false conclusions. The frog has a different context - movement by leaps and bounds. If we consider this phantom multiplier (context), it reduces the meanings of the other signs of the time: mouse - to burrow, to hide; bird - freedom without a cage, victory; arrows - the five tribes (Agathirs, Neurs, Androphages, Melanchlaens and Taurians) which refused to support the Scythians. But the Scythians were going to leap over the territories of these tribes to force them into an armed repulse against the Persians. In pursuit of the Scythians, the Persians would be forced to seize the territories of the refusing tribes and turn them into Scythian allies at war with the Persians. The frog does not mean obedience (earth and water), the meaning of its sign is a battle tactic (attack and retreat).

The key to recognizing the true meaning of the Scythian message was the number of arrows, and it was overlooked by Darius and his wise men as an unnecessary and irrelevant sign, which they replaced with the word "arrows".

Every man is mentally limited. Darius' problem was different - he was a king. For any hierarchical system, the loyalty of the expert to the superior is more important than the expert's professional abilities. Collective (heterarchical) discernment of meaning is inconceivable without equality of participants, regulators and organizers of expert communities. And in this case, the expert opinion reflected only a fear of the necessary freedom of reasoning. It was simply the opposite of the opinion of King Darius 1, without any substantive solution to the problem. The expert might even have wanted to repeat the king's version, but this could not be done according to the regulations of the sages.

Hierarchical systems are capable of turning an expert community into a crowd with the common mind of the most untalented participant (the chief of experts). Heterarchically properly organized and provided with mental tools to support mutual understanding, expert communities are capable of bringing the abilities of all participants to the level of their most outstanding expert.

1.2 Babylon and Rome

Babylon was the first megalopolis in the valley of the Two Rivers with a middle class of small and medium-sized slaveholders. Texts and mathematics served economic purposes. Babylon's competitive advantages in the age of Hammurabi were advanced business legislation, baked on clay by texts, and the irrigation of the Two Rivers to produce rabid crops. Legislation in written form emerges in the second stage of civilizational development, when economic activity becomes more complex, requiring normative regulation and guarantees for the transmission of fortunes by inheritance.

Wealth burns the pockets, and grand new goals are required. In 582 B.C., Nebuchadnezzar, the king of Babylon, had such a goal formulated by his Egyptian wife, Nitokris: it was high time to build a new canal and increase the irrigated area beyond the river floodplains of the Two Rivers. Like we have on the Nile.

They built it. But what happened? The Euphrates began to flow more slowly, and the alluvium began to settle in the irrigation canals. This increased the labor cost of maintaining the irrigation network as it was. Reclamation techniques mechanically borrowed from the Nile to the Euphrates caused the bankruptcy of Babylon. The Babylonian engineers did not ask the Egyptian specialists key questions about the physical similarities and differences between the Nile and Euphrates when they consulted and handed over technical documents. The same words "river," "irrigation canals," had the same engineering context for both groups of experts. Meanwhile, the phantom multipliers were different - the waters of the Nile carry silt, while the Euphrates carries boulders and pebbles from the Armenian Plateau. There were no questions because the hierarch ordered the construction, and the subordinate expertise was obliged to carry out the order without question.

The collective decision-making support in the Senate of the Roman Empire was also untenable. The question of the limits and resource limitations of the centuries-old tradition of constant warfare was not asked in time. The colossal scale of Roman conquests throughout the life of civilization resulted in a catastrophic loss of the gene pool - the best experienced soldiers and warlords. Because of the shortage of civilized recruits, barbarians began to be drafted into service. Society changed, barbarians became part of the elite, and the titular population was required to make huge contributions to the active armies in which the barbarians served. The world was turned upside down. The result was an aggressive mutual misunderstanding and rejection in the elites, followed by the collapse of civilization.

1.3 Prototype algorithm for collective understanding

Methods to ensure the accuracy of Torah copying by scribes can be seen as a network (heterarchical) model of the organizational scheme of collective meaning recognition based on the Hebrew algorithm of collective recognition of grammatical errors in texts.

Rav Mordechai Neugerschl, in the dialogues "Journey to the Top of Mount Sinai," revealed a method to ensure that the Torah text is transmitted accurately to generations without distortion. The rabbi used the example of the children's game "spoiled telephone" for illustration and described a "data protection" algorithm that was able to correct all errors. In this game, when words are passed down the chain of "subscribers," such distortions occur that at the end of the chain the words are different than at the beginning of the game. This is very similar to the change of meaning when passing a text on to other generations.

The Torah scroll was rewritten by humans. It contains over three hundred thousand letters. Each rewrite can introduce errors into the document. After a certain number of iterations, the texts can be very different. The Gospel contains about 140,000 letters. It is about 1900 years old. Yet there are 150,000 versions of the gospel. Moshe's book of Torah is twice as big in characters and twice as old. It has been preserved in its entirety and without grammatical distortion in all Jewish communities.

The Torah scroll was written by Moshe and placed next to the Ark. At that time, the Jews held a gathering of all the people once every seven years at Sukkot in the Temple, where the king read the fifth book of the Pentateuch before everyone, and everyone could hear the exact text and compare it with the one he had in his possession (after correspondence). Ravnaz called the seven steps of the error correction algorithm:

  1. Each participant pronounces a word and passes along with the word to the next participant the object named by that word.

  2. The first participant hands the next participant a note with that word.

  3. Each participant repeats the word to his neighbor fifty times. Even if there are distortions, the frequency of the correct word is sufficient to hear it correctly. If any doubts remain, the participant must ask his or her neighbor again.

  4. Participants are placed in a matrix, e.g. 10x10. All participants in the first row agree with each other on the word that is suggested. If the last participants in each column say the same word, it is transmitted without distortion. It is unlikely that the same error will appear in all rows and columns of the matrix of participants.

  5. All these four actions are performed on ten consecutive days. Participants' state of attention and mood may vary each day. If the same result is obtained on all ten days, there is no distortion.

  6. Participants are motivated by punishment for distortion and encouragement for accuracy of transmission.

  7. Participants are motivated by the importance of the task for society, the state, and humanity.

When all points are met, there is no distortion of data.

Moshe's algorithm is slightly adapted for collective meaning recognition:

  1. Each participant pronounces a key word and distributes to all participants the images (images) associated with that word.

  2. Each participant distributes to all participants the synonymic series associated with the word.

  3. the images and series are coordinated, integrated and distributed to all participants. Each participant remembers the integral images and series by repeating them fifty times. If misunderstanding remains, he/she should ask questions of the trainer and correct the misunderstanding.

  4. Participants are placed in a matrix, for example 10x10. Participants in each element of the matrix should understand the integral images and synonymous series behind the keyword without a single omission or distortion.

  5. All these 4 actions are repeated for ten consecutive days. If the same result is obtained on all ten days, collective understanding of the keyword is possible.

  6. Participants are motivated by rewards and punishments for the same collective understanding of each word.

  7. Participants are motivated by the relevance of the collective comprehension vocabulary task to the community.

If all points are met, collective community vocabulary understanding will be assured.

Successful discussion meetings, for example, require prior agreement on key terms and concepts of the collective brainstorming topic. If the preparation of each meeting were conducted according to Moshe's method of meaning, success and ultimate time savings would also be assured. In unprepared groups starting with five interlocutors, there is not a single word that the entire group understands in the same way [2].

Successful negotiation requires alignment of the interests of the parties with interdependence and interest in the outcome. A strong negotiating position is a low interest with multiple solutions. A weak negotiating position is high interest in the outcome and no alternative solutions. It requires serious preparation of one's negotiating position in one's own discussion session, turning one's weak position into a strong one. Then, too, negotiations can move to the status of a discussion meeting when all parties have strong negotiating positions and are interested in finding a common effective solution. The experts can effectively compare objects and relations (properties) in the following semantic triangles: an object with another object in the sense of some attribute (property or relation); an object with a attribute in the sense of some object or attribute; two attributes in the sense of some object or attribute.

Collective meaning recognition also requires technology to formulate true questions. Asking the right question often means automatically getting the answer as well.

2. Hyperbinary questions.

2.1 Closely related approaches and concepts

Logical semantics is a branch of symbolic logic [4], which studies the relationship between symbols of objects (names, signs) and properties of objects (meanings and meaning as part of meanings). A closely related concept to symbolic logic is mathematical logic. Both use special formalized languages, but mathematical logic applies to the study of mathematical reasoning, while symbolic logic applies to judgments and inferences in natural language.

The creators of symbolic logic are Gottlieb Frege, Alfred Tarski, Rudolf Carnap, and John Kemeny. Their ideas are based on the concept of Gottfried Leibniz, who distinguished between necessary truths (of reason) and incidental truths (of fact). Necessary truths correspond to the laws of logic, while incidental truths are relevant (correspond in a sense) only to the factual state of affairs. Absolute laws of logic are true in all possible (not contradictory to logic) worlds, but laws of fact are true only in some of the possible worlds (ours, for example).

The meaning (relevance) of expressions (as a text or sequence of signs) is given in two ways: logical and factual truth.

The first way (logical truth) - the meaning (context) of a sign is calculated through its frequency (weight). An example is calculating a measure of formal relevance as a comparison of the relevance of a search query with a search answer. Common is the algorithm TF-IDF. The greater the word frequency in the text (TF, term frequency) and less in the set of texts (IDF, inverse document frequency), the greater the weight (rating by two indicators TF and IDF) the answer in relation to the question. The text on this measure is considered more relevant to the search query. The answer will appear earlier (higher in the list) in the search results for that query. "Earlier" is the meaning of the sign (word, text fragment) in the search engine algorithm. Image sounds and images in the calculation of formal relevance is their verbal description. True in the logical approach is all the results of the formal relevance to the accuracy of formulas for calculating the frequency and the convolution of the number of indicators in the ranking. In a more general formulation - logically true statements (texts) are considered true in all permissible interpretations of a given formalized language.

Under the second approach, statements (texts) which are true in some interpretation (in the given sense) but not in all admissible interpretations (universally and absolutely) are considered factually true.

A proposition (text) is factually true if its truth depends on the truth values of its fragments. If a sentence fragment has a change in its content (e.g., context), the sentence can become factually false according to the accepted logical operations. A proposition depends on facts and is factual or synthetic (as opposed to logical or analytic propositions). The negation of a factually true proposition is a factually false proposition and vice versa.

Frege proposed that a proposition be considered analytically true if its proof uses only general logical definitions and laws, and synthetically true if the proof requires the use of propositions of special (outside logic) science. Because of this, Wittgenstein believed that logically true propositions are reduced to tautologies and say nothing about the world.

There are three kinds of semantic correspondence (relevance) to factual truth. For example, with formal relevance, a robot is said to make a decision, even though the person who created the robot (its algorithm) actually makes the decision. Content relevance is determined immediately by a person (SEO-assessor) - how much the answer corresponds to the request. In this case, the result is obtained in fact at once, but only in the sense of a particular assessor. This is like a joker who has received a normal education trying to figure out the questions of the Unified State Exam or IQ tests. The results can make you laugh, declaring the answerer ignorant and stupid. You can only get a good test result by accident, because with an education other than the author's, you can offer dozens of other actually true answers in other senses to these questions or tests. The meaning of the questions is not stated in the tests, by the underthought of the authors of the questions it is believed that it will be a clue. For example, the question "is black white?" can be answered "yes" at least three times, indicating three meanings of the answer in relation to the question (in what sense "yes"): the negative of a photograph, a snowflake drawn with a black pen and a white field as a result of a move from the black field of the chess knight. The picture of the world of two people is already richer than the perceptions and knowledge of each. This statement is the basis of the future collective mind of humanity.

Pertinent relevance - the users themselves are responsible for the truth of the answers. The meaning here is behavioral statistics and the crowd algorithm - if people click on a query to a resource, it means it's relevant to you, too. Or the same, but in a different way - if the crowd is crossing the street, then you need to cross it immediately, even if you were walking in the opposite direction. Pertinence is when the search engine has already achieved relevance, and the next step is how to satisfy the user, who often knows what he's looking for, but does not know how to ask it. So they suggest that he cross the street in droves.

The complete collapse of ideals in the sense of Leibniz's idols has turned out to be the opposite. There is only factual (substantial) truth, but with many meanings and worlds. And the ideal logical truth is trivial. However, Leibniz also called logical truth absolute - one with which everyone agrees (tautology according to Wittgenstein).

For this Frege also paid a very painful price - immediately after the publication of the second volume of his magnificent work, he received a one-page letter from Bertrand Russell, which shocked all mathematicians and created a crisis in mathematics for the past 120 years. Russell began by praising Frege's work and expressed his absolute support for the author. "But I found a slight difficulty," Russell stated absolutely ruthlessly. The slight difficulty turned out to be one of the axioms on which Frege based set theory-the axiom of selection.

Frege noted in his book: "Hardly anything worse can happen to a scientist than to have the ground knocked out from under him the very moment he finishes his work. This is exactly the situation I found myself in when I received a letter from Bertrand Russell when my work was already completed.

The mathematical theory of categories, discovered in 1945 by Samuel Eilenberg and Saunders Mac Lane, deals with this fundamental problem.

3. Hyperbinary Algebra of Questions

3.1 Question Logic

One of the sections of symbolic logic is question logic [5]. A question is a statement on the basis of which we need to find another statement so that together they constitute a complete answer to the question posed.

A question is a part of the answer and just a fragment of the text. The question is a form of definition of knowledge of ignorance. The main thing in it is semantics, meaning. It does not matter at all what punctuation mark stands or does not stand in place of the question mark. For example, it is not required to put a question mark in the line of a search engine.

It is more important that the question is necessarily preceded by another question and the past complete answer (context) or a sequence of complete answers (reasoning). The question can be posed in relation to any fragment of the text.

Questions and answers must necessarily be connected (relevant) in meaning (context). Otherwise, the answers will not refer to the posed question, but to another possible question. In the logic of questions (erotic logic) it is believed that any question (meaningful or meaningless) can be answered in a way that is relevant to it, but not the only one, and with a zero measure of the meaningfulness of a meaningless question. But this only means that not all contexts are involved (found or investigated).

A question is considered logically correct if there is at least one correct answer and logically incorrect otherwise.

A correct answer is such an answer to the posed question, which is relevant to it (corresponds in the specified sense) and at the same time is a true statement or reasoning.

An incorrect answer is an answer which, although relevant to the question posed, is not a true statement or reasoning.

It seems appropriate for text algebra to change the concept of relevance of questions and answers.

3.2 Definition

A question is true if the answer is binary and singular (true or false); otherwise the question is false.

In other words, a question must be phrased so that it can be answered affirmatively or negatively. Then the wording of the question is considered worked out (true). If it cannot be so answered, then the question is not good enough (false).

In question logic, a complete answer is the concatenation of question and answer. Here (actually) a true question is both a question, an answer, and a sense in which the answer is relevant to the question.

For example, the question "where does the Volga River flow into?" is false. It is impossible to give an affirmative or negative answer. But let the choice be offered: Caspian Sea, Kama River. The question is still false. The question is binary (you have to put a yes or no label). But one cannot answer in the affirmative twice to two (seemingly) contradictory questions.

One more component, the "meaning of the question," is missing in order to achieve the missing unambiguity of the answer. This meaning made it possible to make the answer not only binary (yes or no), but also unambiguous (yes). In this case, all four true questions will already be different:

Does the Volga River (geographically) flow into the Caspian Sea? No.

Does the Volga River (historically) flow into the Caspian Sea? Yes.

Does the Volga River (geographically) flow into the Kama River? Yes.

Does the Volga River (historically) flow into the Kama River? No.

All four questions are (factually) true, but became so only after they were formulated to take into account the missing meaning of the key word in the question. The word "flows into" has two contexts (meanings): geographic and historical. On the geographical map, the stream flows into the river, but not vice versa. At the confluence of the Kama and Volga, the Kama River is more significant on the map as a watercourse than the Volga. But it has been accepted (historically recognized) that the Volga is more significant (mother) for Russia than the Kama (not the mother).

Therefore, geographically the Volga was absorbed by Kama and further Kama flows geographically into the Caspian Sea, while being called historically the Volga. But then the Volga only historically flows into the Caspian Sea.

For example, the Oka River flows geographically into the Volga, because just upstream of the Volga in the confluence area, the Volga River is more significant on the geographical map than the Oka. At the same time, if the Oka flows into the Caspian Sea, it flows only indirectly, like other rivers and streams flowing into the Volga.

The ultimate redistribution of content from answers to questions is motivated by the changing role of questions in the modern world. A well formulated question with the help of modern information technology is guaranteed to get the right answer. The problem is the formulation of the question. Therefore, the modern teaching paradigm must recognize: the ability to ask questions today is more important than memorizing answers (as it was in the past). The main purpose of a teacher (or better, a personal robot mentor) is to teach the skills of question formulation or problem-setting through examples, systematization of answers, and the next stage of question formulation. The student must demonstrate the ability to ask true questions when testing. The skill of decomposing a primary question into elementary questions must be mastered. The questions, not the answers, should be graded. The relative number of true questions determines the overall grade.

The question requires semantic markup, specifying all the contexts, making meaning chains, unfolding and revealing its content, how to create an abstract from a text title, an article or book from an abstract.

3.3 Hyperbinarization

The original question about the Volga was false, but useful for subsequent semantic parsing by concepts and bringing it to a true question. True questions don't come out of nowhere; they are the result of solving the problem of meaning recognition.

In text algebra, all words of the original question, represented by matrix units, have phantom multipliers of all possible contexts of each word. The addition of such matrix units with phantom multipliers into a matrix hyperbinary question is concordant addition. The total phantom multiplier of the matrix question is the reduction of all phantom multipliers as the product of the phantom multipliers of all matrix words of the hyperbinary question.

The product of phantom multipliers in the simplest case is the intersection of their dictionaries. If, depending on the chosen text coordinating rule, the co-multipliers of the product of phantom multipliers have the same indices, the procedure of matching the common phantom multiplier is a bit more complicated and will require to solve the system of binary equations (7.3.7) from [1].

When calculating the intersection of matrix contexts (phantom multipliers) a list of all mergers of the Volga with other rivers and reservoirs will appear. Since for any pair "Volga - other river or body of water" the word "inflows" has the meaning (context) "flows in", "enters", or as an antonym "includes", the common intersection will be the only object - the river Kama.

4. Hyperbinary evaluations

4.1 Close Approaches

Frege's semantic triangle illustrates the relationship of three concepts - sign, meaning, and meaning. It implicitly assumes that the order in which the properties in the list of meanings and part of these meanings (sense) are listed is immaterial. Generally speaking - it is not. Gone is the significance (importance, value) of the properties. But if we assume that some properties may have a different value, for example, when choosing a car or staff selection, it is the order in which the properties are listed (rating) is in some cases more significant than the list itself. For example, a list of car properties (price, fuel efficiency, clearance, top speed, etc.) is the same with the accuracy of measurements for all cars in a car dealership. And such disordered lists will not give the buyer anything. But if the prices, for example, are presented in ascending order, it already facilitates a choice essentially. If the customer has defined the meaning of car buying, then adding "color" property to the list of car meaning, he/she will spoil everything, if the attractive price and favorite color do not coincide. If you add a few more required properties to the selection task, you may never buy a car at all.

The car, employee, or supplier selection task are the most common illustrations of numerous decision support methods. One of the earliest methods in the early seventies was developed by Thomas Saaty, the Hierarchy Analysis Method (HAM) or otherwise the eigenvector method [6]. The basis is to coordinate the subject area using a scale of measures of preference (importance) in decision-making. For example, it can be a nine-point rating scale:

1. Equal importance - both properties have equal importance.

3. Moderate superiority.

5. Substantial or strong superiority.

7. Significant superiority.

9. Very Significant Superiority.

2, 4, 6, 8 are intermediate degrees of superiority, with values falling between the significance scores defined above.

The second basic concept of the MAI is the comparison matrix. The properties (solutions and criteria as elements of hierarchies) are compared in pairs by the expert and the corresponding matrix (positive inverse symmetric) is filled with numbers and inverse numbers to those numbers (the latter is monstrous in terms of mathematics) from the scale of scores. The inputs are the decision/criteria matrices. The pairwise comparison matrices are the decision/resolution matrices for each criterion. The number of comparison matrices is equal to the number of criteria plus the matrix of pairwise comparisons of the criteria themselves.

For each matrix of pairwise comparisons a vector of local preferences with elements - geometric averages of products of evaluations of each matrix row followed by normalization is constructed. In this case all matrices of pairwise comparisons should be matched with a given accuracy. Ideally, consistency is the fulfillment of the transitivity condition a_{ik}=a_{ij}a_{jk}for all elements of the matrix a_{ik} of comparisons. The expert is bound to be wrong, and the size of the errors is called the consistency index. Saaty, based on numerous and impressive author applications of his method, developed guidelines for acceptable error values for the consistency index. If the error value is exceeded, the expert is asked to repeat the estimate in the corresponding part of the matrix of pairwise comparisons.

The local decision priorities are multiplied by the priorities of the corresponding criteria and summed over each element according to the criteria. As a result, the global priorities of decisions are determined, taking into account the priorities of the criteria themselves. The highest ranking will correspond to the solution with the highest global priority value.

After inventing the method of hierarchy analysis and demonstrating its success, Saaty attempted to justify this method for more than forty years. The point is that the objects of coordination in the construction of scales of estimation are not numbers, but numbers of estimations. They look like numbers, but they are simply names (item numbers). You can't do arithmetic operations with the numbers. It makes no sense for a budget to add up the numbers of bills. And in Saati's method, the division operation appears immediately when the matrix of pairwise comparisons is constructed (the matrix is inversely symmetric). Of course, Saaty understood the contradiction in the basis of the method.

In his theory of the importance of criteria [7] V.V. Podinovsky immediately gets rid of this contradiction. A grade of "excellent" is not considered 2.5 times better than a grade of "unsatisfactory". The grading scale is also present. But its use is limited to the study of order. Such a scale Podinovsky calls ordinal or qualitative. For example, by analogy with the numbering of houses in a street, one can conclude that house number five will be three houses away from house number two, but one cannot conclude that it is two and a half times further. Podinowski's theory uses vector grades for solutions, where the components of the "vectors" are the numbers from the grading scale. A terminological clarification is needed here. The complexes of grades are not vectors. If it is impossible to perform arithmetic operations with grades, then such objects are not even the "weakest" of vectors - arithmetic vectors. The term "tuple" is more acceptable. In a tuple its elements are ordered and this is sufficient for the choice of term.

Also important for the scale is the following requirement of homogeneity: each gradation of the scale must reflect the same level of preference for each of the criteria. This condition is met, for example, when the gradations are verbal (verbal) for all the criteria, for example, "excellent", "excellent", "satisfactory", "disgusting".

Podinowski's method consists in dividing decision scores into three classes: dominant, non-dominant (Pareto-optimal), and indifferent. Accordingly, the ranking of solutions according to the given solutions can be constructed for the first class on the basis of a component-by-component construction of the corresponding chains of tuples of estimates satisfying the condition of transitivity.

For quantitative evaluations Podinovsky proposes to decompose the criteria into equivalent parts, then we can already perform arithmetic operations with the evaluations of such parts. For evaluation of students it means the necessity to divide academic subjects into such equivalent parts so that the credit on some section of quantum mechanics, for example, corresponds to some credit on physical education. Then the total normalized graduate grade will be calculated correctly.

4.2 Hyperbinarization

The hyperbinaric approach to decision support methods can be demonstrated with the example of Podinowski's assessment of student performance [7]. There are seven students and four academic subjects. The grading vocabulary is five points.

A matrix grade is understood as a matrix unit with the first index being the course number, the second index being the grade number. For example,E_{3,2}is a "D" in the third subject. The hyperbinarization of grades would be both similar to the matrix Morse code by coordinating by rule 2 and the approach to mathematical texts, where elements of formulas are proposed to be treated as signs. The latter is very appropriate to grades, which are simply numbers of types of grades (excellent, good, satisfactory, unsatisfactory) and to which arithmetic operations are not applicable. Grades are simply texts.

The hyperbinary grades of Podinovsky's students:

  1. (E_{1,3}+E_{2,5}+E_{3,5}+E_{4,4})(E_{3,3}+E_{4,4}+E_{5,5})

  2. (E_{1,4}+E_{2,4}+E_{3,4}+E_{4,5})(E_{4,4}+E_{5,5})

  3. (E_{1,5}+E_{2,4}+E_{3,3}+E_{4,3})(E_{3,3}+E_{4,4}+E_{5,5})

  4. (E_{1,3}+E_{2,5}+E_{3,3}+E_{4,5})(E_{3,3}+E_{5,5})

  5. (E_{1,4}+E_{2,2}+E_{3,4}+E_{4,5})(E_{2,2}+E_{4,4}+E_{4,5})

  6. (E_{1,3}+E_{2,5}+E_{3,3}+E_{4,5})(E_{3,3}+E_{5,5})

  7. (E_{1,5}+E_{2,3}+E_{3,4}+E_{4,3})(E_{3,3}+E_{4,4}+E_{5,5})

The fourth and fifth students are the same in their grades. The expressions in parentheses to the right are the right phantom multipliers (fragments of the grade vocabulary) - these multipliers determine the subgroups of underachievers (#5), triplets (#1, #3, #4, #6, #7), achievers (#2), and honors students (none). The most numerous part of the students is the "C" students. Classifying them requires improving the grading scale by fixing preferences. For example, "C" students are those who have a grading pattern (E_{1,5}+E_{2,4}+ \ldots + \ldots), taking into account that the first subject is more important than the second. Then an excellent student out of the failing students is the third student.

In the case of binary grades (pass-fail) in hyperbinary grades, the second indexes have numbers 1 or 2 (as in Morse code), the first index denotes the number of pass. Total student grades for the entire list of credits, into which all exams are divided, are determined by the formula (5.2.9).

All algebraic operations with some features of addition (2.3.4.1) are applicable to hyperbinary grades, in contrast to "vector" grades, to which no single operation can be applied. Moreover, if the grading scale itself is verbal, then the corresponding hyperbinary grades as sums of matrix units are already mathematical objects.

The division of learning items into equal credit units is analogous to the hyperbinary reduction of questions into true questions.

In the general case for grades it is necessary to convert the additive form of hyperbinary numbers into a multiplicative form, as in (5.2.1), (5.3.1), (5.4.2), for which the common multipliers will be the Gröbner-Shirshov basis, similar to the reduction of commutative polynomials [8] via "column division". Hyperbinary numbers (estimates in this case) are also polynomials, but noncommutative. The division "in column" is differentiated on the left and on the right. The Gröbner-Shirschov bases (common multipliers of hyperbinary grades) are classifying signs of student performance. The left-hand bases are attributes of classification by criterion importance, the right-hand bases, are attributes of classification by grade importance.

5. Hyperbinary philosophical categories

Structuring, summary, contextual vocabulary, and classification (correspondence to some section-title, UDC for example) are important for text comprehension. The highest (universal) headings of natural language texts are philosophical categories. In the history of philosophy, each category is a masterpiece, a piece of merchandise, created by the "handiwork" of the most outstanding scientists. Information technologies have not yet been used in the creation of categories. There are about a hundred known logical, ethical, and aesthetic categories. It is extremely attractive and useful to calculate the highest headings of the natural language corpus (semantic categories) and compare them with the list of categories created "by candlelight".

The headings are the names of directory boxes into which all the words of natural language can be put. In this case, the defining property of the contextual language is the ability to stack one word in several directory boxes. A paradoxical situation arises when the names of the boxes of the directory can be more than the number of words of the language.

For example, one word "rectangle" can fit into three catalog boxes. The generic box names are quadrilateral, parallelogram, and polygon. Among them you can specify the nearest genus. For a rectangle, the closest genus is a parallelogram. The closest is the one with which there are more properties in common. In addition to "number of angles," the second phrase is "number of equal angles."

Two words can actually be similar if there is a third word with which they are both simultaneously similar. For example, such a third word for rectangle and quadrilateral is polygon. Rectangle and quadrilateral are similar in the sense that they are both polygons. But a rectangle and a polygon are not similar in the sense of a quadrilateral, because not every polygon is a quadrilateral. Similarly, we can define the relationship of difference. Two words can actually be different in the sense of a third word if at least one of them is not related (does not belong to that subset) to that third word. For example, a parallelogram is distinct from a quadrilateral in the sense of a square. Not every parallelogram and quadrilateral has equal angles and sides.

Two words are connected by a relation of subordination (nesting or hierarchy) if they are connected by a genus relation. Genus is subordinate to species because species has all the properties (definition words) of genus, but species also has additional properties as compared to genus. For example, square, is a generic word in relation to the word "rectangle", because it has all the properties inherent in a rectangle, but also the property of equality of sides.

A sense is that third word or subset of words to which the two words, between which the relation is established, belong. For the multiplication of the catalog of words, their meaning is responsible.

The catastrophism associated with the fact that there may be more catalog box words than words of the language itself is imaginary. In any natural language, there are special saving words, called philosophical categories, with which the bad infinity of catalog boxes can be limited. The reason is the lack of meaning in the philosophical categories. These categories (if each is considered in isolation from the other categories) have, as logical concepts, zero content and maximum scope.

One paradox (words with zero content) removes another problem related to the paradoxical notion of the catalog. One formulation of Russell's paradox (upsetting Frege): "Catalogs are books that describe other books. Some catalogs may contain other catalogs, and some may even describe themselves. Is it possible to make a catalog of all catalogs that do not describe themselves?"

For the system of philosophical categories of natural language, there is no ontological problem of a catalog.

5.1 Philosophical Categories

An outstanding and not quite utilitarian-appreciated discovery of European civilization (not used in applied tasks and economic activity like mathematics) is Georg Hegel's development of a method for investigating any subject area based on the logical continuity of meaning in the language of categories (philosophical - a necessary clarification, because mathematical categories appeared in 1945).

Logical categories (as Hegel called philosophical categories) are universal and universal concepts, defined one through another through relations (connections) and have no genus (a more general concept). The relations themselves are also universal and are universal laws for any subject area. The world constants of understanding (patterns of category logic) are used minimally consciously or maximally intuitively by any reasonable person to study the changing world and adapt to it.

The content of categories is minimal and is determined only by the number of inter-category connections. The volume of categories as concepts is maximum. This means that each category can be interpreted by all words of natural language or terms of special language. But each word can also be assigned a set of categories with an indication of the significance of each category for the meaning (interpretation) of that word.

Categories are usually interpreted by words and terms, for example, synonymic series as relations of equivalence and tolerance (similarity). Or as relations of difference and opposition (antonymic series). There are two more kinds of relations - of belonging and of order.

Examples of definitions of categories through categories: quality is an essential property, quality is the specificity of a thing, quality is absolute, property is the relation of the singular, property is relative, property is an object, relation is a special property, relation is the property of the general, relation is the general of objects, object is a whole of properties, the whole is the form of relations of parts, object is singular.

Examples of interpreting the category of the universal through synonymic series: boundless, everywhere, always, universal, all-embracing, universal, nationwide, comprehensive all-encompassing, comprehensive, all-encompassing, all-encompassing, everywhere, every, global, profound, unified, regular, exhaustive, every, any, world, nonunique, unspecific, vast, common, general, pandemic, universal, pervasive, ubiquitous, total, complete, perfect, joint, total, essence, total, universal, frontal, holistic, broad.

Here, some of the interpretive words are themselves categories, e.g., general. This is why categories are connected and defined through categories. For example, the universal is the absolute abstract common. This formula is derived by using the corresponding synonymic interpretations of the categories "equivalence," "similarity," "absolute," "abstract," and "general. The repeating words in the synonymic rows of categories are used to establish connections between categories.

Some interpretive words are close (in relation to similarity) to other categories, for example, boundless (infinite), continuous (continuous).

Interpretation of categories is possible through antonymic series. The antonyms of the universal are the singular, the local, the local, the particular.

Part of the interpreting words are pronouns ("instead of the name"). These are the historical antecedents of the categories. People in the formation of speech as a second signaling system used these category substitutes. The pronouns bind the categories in an excessively rigid way. There are fewer pronouns than categories, but they are part of the synonymous series of all categories. At the first stage of cognition, rigidity is appropriate. For example, it is easier to understand the meaning of the combinations of the categories absolutely abstract (some kind of everywhere) and abstractly absolute (some kind of everywhere).

In middle school, one of the exercises is to replace words in a text with pronouns so that the meaning of the text is retained and the text is recognizable. Mankind would move to a qualitatively different level of development if schoolchildren were given such tasks for categories.

Two hundred years ago Hegel created his language as a supranational, universal and universal language of the human population. "The greatest impudence in presenting pure nonsense, in a set of meaningless, wild combinations of words," described this language by Arthur Schopenhauer, Hegel's colleague at the University of Berlin.

At present only a few adults understand this language. E.V. Ilyenkov and P.G. Kuznetsov mastered the language and Hegel's method masterfully. Ilyenkov used this knowledge in A. I. Meshcheryakov's brilliant result-oriented project to teach deaf-blind children in a boarding school in Zagorsk (1963) [9]. Kuznetsov applied the method in numerous applied tasks (development of life support systems, goal-oriented management, in physical economics).

Children and old people are quite proficient in the language of pronouns (categories) on an intuitive level. Having grown up, having accumulated a logically unstructured vocabulary of ten thousand words, people stop using a hundred basic words-categories (they do not differ from other words in importance for them). Only in old age, as a way of dealing with dementia, do some people consciously return to categories or unconsciously to pronouns.

As amazing a fact as the possibility of teaching deaf-blind children is children's linguistics. Children forced to communicate with people of different ethnicities (e.g. in refugee camps or immigrant communities) form languages of an abstract type called Creole. All Creole languages share common structural features that reflect the categorical basis of the language. All over the world Creole languages with different base vocabularies (English, French, Spanish) reveal the same homogeneity, the same grammatical structures and categories [10].

О. Espersen argued in "Philosophy of Grammar" that along with syntactic categories, which depend on the structure of each language, there are also extra-linguistic categories, which do not depend on more or less casual facts of existing languages. These categories (conceptual categories) are universal because they are applicable to all languages, although they are rarely expressed in these languages in a clear and unambiguous way [11].

The system of logical categories can be represented as a set of six comparison tables. Each of them uses one of the six types of category relations (equivalence, tolerance, belonging, difference, opposition, and order). In each table, the column headings and row names are categories and their combinations. Let there be sets of interpretative words for each category and each of the possible category combinations for each of the six relations. A relation between a category (combination of categories) and any combination of categories (category) exists if their sets overlap (there are common elements). Then the zeros and ones in the cells of these tables mean the presence or absence of connections between categories. This is Hegel's logical system as an absolute idea.

At the same time it is possible to perform the reverse operation. Express the words of interpretations through categories. To make a categorical parametrization of words and terms. Parametrization - because categories are many less (tens), terms are thousands, words are tens of thousands. In information retrieval and storage tasks this parametrization is called reverse indexing. With reverse indexing, a logical model for finding and storing words and text fragments will be automatically constructed.

In formal (mathematical) logic, one can use its laws to find contradictions or solutions by simplifying predicates or statements. Thinking can be replaced by calculations.

Hegel's logical system, like crutches for the mind, helps to find concrete meaning where it is unknown, but can be an unexpected discovery. Although everything at first looks like an obvious contradiction. Even the 5=6 contradiction can be resolved by specifying an appropriate combination of the categories "number of possible measures" in the sense of acceptable error in approximate calculations.

If there is a unit in the table as an established relation between categories and their combinations on six kinds of relations, then there must also be a concrete meaning to this combination in the corresponding subject area. Then it is possible and necessary to assign some subject content (a set of definable concepts) to this cell of the table. It's like in Mendeleev's table: if there is a cell, there must also be a chemical element, even if it has not yet been discovered. Hegel called this process the ascent from the abstract to the concrete.

The purpose of Hegel's system (logic of meaning or content) is to prepare the subject area for the use of mathematical logic. Name objects, define their properties and relations. Fill the cells of the category table with meaning (key words of the subject area). This will allow further numbering of objects and arrangement of existence and generality quantifiers in logical functions of formal (mathematical) logic.

In Hegel, this process is called the removal of contradiction. In this case, the filling of the table of categories (preparation for calculations) takes place iteratively, in cycles (repeated ascent from the abstract to the concrete to the necessary or admissible detailing), by colliding the categories of the singular, the particular, the general and the universal with other categories and relations. This is the essence of Hegel's discovery - the creation of the logic of content (meaning) as an independent science and its additionality to the logic of form (formal logic).

Research on the categorization of various subject areas was supported by grants [12], [13].

5.2 Hyperbinarization

The incomplete list of philosophical categories created "by candlelight" consists of three sections:

A collection of 60 logical categories.

Absolute, Abstract, Infinite, Thing-in-itself, External, Internal, Possibility, Time, Universal, Movement, Actual, Discrete, Singular, Uniform, Natural, Other, Law, Ideal, Artificial, Quality, Quantity, Finite, Concrete, Matter, Measure, Necessity, Continuous, Nothing, Nothingness, General, Object, Special, Distinction, Relative, Relation, Order, Subject, Sign, Belonging, Reason, Space, Opposition, Difference, Reality, Consequence, Randomness, Content, Medium, Subject, Substance, Existence, Essence, Similarity, Identity, Form, Whole, Purpose, Part, Equivalence, Phenomenon

A set of 10 categories of ethics:

Faith, Goodness, Confusion, Evil, Joy, Conscience, Justice, Suffering, Happiness, Betrayal.

A set of 10 categories of aesthetics:

The ugly The sublime, The impressive, The comic, The inferior, The poetic, The beautiful, The prosaic, The indifferent, The tragic.

The categories constitute a system. This means that categories can be defined through each other as categorical textual formulas because they have common words in their synonymous rows of interpretations that contain, among other things, categories. Examples of categorical formulas are: Whole - One and Many; Happiness - Relation of the Outer to the Inner, Absolute Absolute - Nothingness (the ultimate generalization is its absence); Abstract Absolute - Nothingness (the generalized ultimate measure is indefinite existence), Absolute Infinite - Whole (the ultimate limitless is the undivided, all whole), Infinite Absolute - Matter (the unlimited limit is substance).

The defining property of the words-categories is the maximum volume of word-contexts for the corpus. At the same time, the categories themselves are not distinguished by a great frequency of direct use in texts [11]. Categories cannot be recognized with the help of frequency methods even in philosophical texts. The pronouns, semantically distant relatives of categories, differ precisely by the frequency of use. Therefore, historically, categories have been defined in long and difficult reflection ("by candlelight"), and recognition automation has not been used.

In text algebra, categories are represented by a hierarchical system of nested (fractal) phantom multipliers - what are called refined contexts. A refined context is when each word context is also represented by its phantom multiplier (second-level context) and so on down to the roots of the semantic tree of the genus-species hierarchy.

Let a supposed candidate word for a category correspond to some set of matrix words taken as a context of this word (phantom multiplier). If the category itself is not obliged to have an appreciable frequency in the corpus of the language, its phantom multiplier as a fragment of the text already has a chance to be repeated. If we further consider the refined context, the frequency for the category-applicant must increase. If it does not increase, the word is excluded from the category contenders. And the process begins with a new challenger.

If the frequency increases monotonically, the limit should be expected on a refined context of one percent of the number of words in the corpus of language. As an initial approximation, it is advisable to take a set of one hundred units of classical philosophical categories "by candlelight".

The algorithm is iterative. For the convergence of the computational list of categories it is advisable to establish horizontal (heterarchical) connections at each iteration. In this case we can expect a uniform partitioning of the language vocabulary into categorical fragments.

6. Mentoring

Learning is based on motivation. The sources and form of motivation can be different. In Meshcheryakov's experiment of teaching deaf-blind children the initial motivation was possible only on the basis of taste and tactile perception of the world. This was the beginning of learning, and then some boarding school graduates were able to enter and successfully graduate from MSU. In the language of philosophical categories, the child was a "thing in itself" in the beginning, and the teachers helped him to realize that there was "something" and "other" (the world surrounding the child). E. V. Ilyenkov was A. I. Meshcheryakov's supervisor during his studies at the Philosophy Department of Moscow State University.

I heard a personal story in the seventies from a graduate of the Zagorsk boarding school through a typhlo-syllabic interpreter. As an infant, the future doctor of sciences liked sweet tea. For deaf-blind children one cannot be late with mental help - after a certain point it is impossible to get the child out of the abyss (it turns into a plant and dies). Teachers with a motivational spoonful of tea managed to help a deaf-blind child understand that the world exists and build his grand mental picture of the world, more meaningful and systematic than that of most people. This is a masterpiece of pedagogical skill that still seems fantastic.

Equally fantastic so far is the creation of a personal digital teacher.

6.1 Mental Cloning

Extended reproduction of the collective mind requires a paradigm shift in education. The defining characteristic of the mentor-protégé, teacher-student relationship is the formula of "many to one and one to many. In the necessary continuous learning throughout biological life, each participant of the collective mind should be provided with a personal virtual collective mentor (a technological clone of the ideal teacher), and the learner himself, in turn, should become part of the collective mentor.

The systematized and structured contexts of words that people learn are called the logical, ethical and aesthetic components of the human model - its phenotype. Human beings are their genotype and phenotype. In the past, they were biologically stored in a person's personal genetic sequences and memory. Modern information technology makes it possible to store not only the genotype, but also the phenotype separately from the person. It is possible and necessary to preserve and accumulate (update) the phenotype in learning throughout the biological life. The learning phenotype is used, tested and refined in the processes of understanding meaning. The individual is objectively motivated to create his or her second self. The created formalized phenotype will allow for human immortality by biological cloning of genotype and phenotype, or/and virtually in information space.

If you create your mental clone as a set of personal dictionaries and classifiers of your knowledge and rules, interests and motives, you can construct models of mentor (mentor) or protégé as a composition of dictionaries.

Based on an analysis of the properties of real protégés and mentors, the relations of many objects to many are decomposed and then combined according to the target function. One protégé can learn from many mentors by synthesizing their knowledge. One mentor can train many protégés. Protégés and mentors can switch roles. The correspondence of real and virtual verbal images needs to be constantly verified. This is the main methodological requirement of the new educational product - the study of one's digital image, its accompaniment and development for the purpose of self-development and self-education.

Search for partners (selection of components of the digital mentor) for learning can take place on the basis of computable titles, summaries, definitions of concepts, key semantic fragments of knowledge description, skills, compared with the texts describing the learning objectives.

6.2 Hyperbinarization

The main tool of the digital human model can be hyperbinary Nether chains, which are a mathematical explication of the philosophical categorical pair "part-whole". Each new knowledge must be synthesized into the existing system of knowledge. The tool of semantic synthesis is the algorithm of additions to the Neuther chain of meanings, which can substantially change the entire previously ordered construction. A constant check of the transitivity of knowledge fragments (hyperbinary elements of the Neuther chain) and, if necessary, its correction are required.

Navigation through the language corpus can be realized on the basis of calculation of the relevance of hyperbinary semantic (phantom) n-grams.

The creation of a synthetic digital mentor requires appropriate personal association of knowledge in a heterarchical (network) system. The unification of meanings can be performed through concordant addition of the corresponding phantom multipliers of personal contextual dictionaries.

Motivation for mental development can be based not only on utilitarian reasons, but also on egalitarian ones. When a schematic personal picture of the world has already begun to take shape in a person, its further development may be based on aesthetics. It is just as in mathematics-its development has not been as economically motivated as in Babylon for a very long time. The main thing is the beauty of logical consistency and the ability to explain and calculate much through a small number of concepts.

References

  1. Pshenichnikov S.B., Algebra of text // Researchgate, 2022 - 117 p.

  2. The Mind Map Book: Unlock Your Creativity, Boost Your Memory, Change Your Life / Tony Buzan, Barry Buzan 2010.

  3. Solomonik A.B. Philosophy of sign systems. — Мoscow: URSS. 2020. 408 p.

  4. Symbolic Logic: Textbook / Ed. by Y.A. Slinin, E.F. Karavaev, A.I. Migunov. St. Petersburg: Publishing house of S.-Petersburg State University, 2005.

  5. Belnap N., Steele T. Logic of questions and answers. Moscow, Progress, 1981. 290 p.

  6. Saati T.L. Decision Making. The method of hierarchy analysis. Moscow: Radio and Communications, 1989.316 p.

  7. Podinovsky V. В. Introduction to the theory of criterion importance in multicriteria decision-making tasks. Moscow: Fizmatlit, 2007. 64 p.

  8. Ardjantsev I. V. Groebner basis and systems of algebraic equations. — Moscow: MCNMO, 2003. 68 p.

  9. Meshcheryakov A. I. Deaf-blind children. Development of the psyche in the process of behavior formation. Moscow, Pedagogics, 1974. 327 p.

  10. Bickerton D. Adam's Language. How people created language. How language created people. Moscow, Languages of Slavic Cultures, 2012. 336 p.

  11. Jespersen O. Philosophy of Grammar. Moscow, URSS, 2006. 404 p.

  12. Pshenichnikov S.B. et al. Metric and set-theoretic methods for synthesizing and analyzing meta-descriptions of data warehouses. RFBR 13-07-00659, 2013-2015.

  13. Pshenichnikov S.B. et al. Development of a universal categorical standard and metalanguage for presentation and storage of educational content data and practical implementation on an educational Internet portal. FGBU "Foundation for Assistance to Small Innovative Enterprises in Science and Technology". 11478р/20902. 2013.

Теги:
Хабы:
Всего голосов 2: ↑2 и ↓0 +2
Просмотры 677
Комментарии Комментировать