SAPIENS

Saliency and Attention: rePresentation, Interpretation and EmergeNce

Attention is a complex cognitive function essential for explaining human behaviour that allows us to select the most relevant events or items in our environment in order to focus our sensory and cognitive resources on them. It can be modulated either by bottom-up sensory-driven factors, or top-down task-specific goals. In the former case it is also referred to as salience or saliency.

Understanding saliency and attention is a highly challenging scientific endeavour and creating artificial machines capable of imitating them, a remarkable technological step forward. Despite the substantial advances in the field, including our own under our previous MinEco funded project SAMURAI (Saliency and Attention: MUltimodality, context-awaReness, self-Adaptation and bio-Inspiration) and others, the challenge is still far from being overcome.

In the meantime, two prominent technologies spanning this and several other disciplines have caused a profound impact on this research agenda: the deep learning paradigm in machine learning, and the maturity of sensor devices. On both these lines our research group has accumulated significant expertise. In this light we have identified the following key directions for advancing this technology:

1. Representation. How to measure and describe attention at various levels of detail is a hard problem due to the limited ability of the measuring devices to capture the phenomenon and inter-subject variability. From the data modeling perspective, this problem is two-fold: first, there is an underspecification of the target labels and second, a lack of appropriateness in input features, specially to model the dynamics and multimodality of phenomena. In this line, our proposal with SAPIENS is to explore unsupervised, weakly supervised and multilabeling methods with an emphasis on the representation of multimodal streams to take advantage of cross-modal synergies. This is fully aligned with the open lines of research in the machine learning community and therefore current methods are likely to evolve in this direction.

2. Interpretation. With the advent and spread of deep learning techniques, a growing concern is their lack of interpretability, in essense, their being conceived as black-boxes. This undermines their applicability, on the one hand, as a tool for scientific understanding of phenomena and, on the other, as an aid for experts in which they are likely to be used as modules inside a more complex system, possibly including human interaction in the loop. SAPIENS will adopt the Exploratory Data Analysis framework, explore information-theoretical evaluation methods for their characterization and search for bioinspired mathematical models behind current state-of-the-art machine learning methods.

3. Emergence. The emergence of order and organization in systems composed of many autonomous entities is a very basic process but very difficult to model or explain. In the phenomenon of attention we observe how a complex task-driven response arises from low level sensory inputs. In SAMURAI we demonstrated the capability to acquire knowledge through the discovery of latent classes or topics in top-down visual systems but not with aural or multimodal data streams. Within this wider framework SAPIENS aims at providing a more generally applicable solution making use of the wealth of scientific results available on this matter.

SAPIENS considers several real applications to test the theoretical advances in these three directions.

Conclusiones

At SAPIENS we intended to investigate the cognitive processes of human attention and salience to obtain computational models, following in the footsteps of our previous SAMURAI project. To do this, we designed three main axes for progress: representation, interpretation and emergence.

  1. In the first of them, the search for better representations for modeling visual and auditory attentions adopted the paradigm shift in representation and feature extraction brought about by deep learning and focused on finding high-level features, either through hierarchical models or the representation of the appropriate contexts through Capsule Networks. On this, we generated salience models based on the so-called echoic memory and recurrent networks for the case of the auditory modality and latent topics for visual space-time attention. For this purpose, pre-existing data sets have been used, three of them have been annotated with an eye tracking device and two new ones have been recorded.
  2. In the second axis, we have made important advances in the task of exploratory data analysis based on k-FCA (Formal Concept Analysis) and in the development of methods based on information theory for the interpretation of the behavior of machine learning methods, having laid the theoretical foundations. Extensions have also been developed to alleviate the blind spots of this algorithm. In addition, we have postulated the connection between human perception and cognition with informational transformations of classical deep learning methods. Finally, we have applied interpretable deep learning methods to melanoma detection that facilitate diagnosis. In this objective, other lines of research related to affective computing and tele-education have also been opened.
  3. Finally, in the third axis, which we consider the most complex and long-term, many of our investigations are still open: thus, we have investigated the integration of unsupervised models of salience in supervised systems based on deep learning and shown their viability in a multimodal task. The advances we have achieved in understanding computational awareness and even, in an incipient way, of consciousness, we consider to be reasonable for the level expected within the framework of SAPIENS and have given rise to the request for several international projects.

Publications

(Open access in our Institutional Repository: https://e-archivo.uc3m.es/handle/10016/1591)

  • [DOI] M. Molina-Moreno, I. Gonzalez-Diaz, J. Sicilia, G. Crainiciuc, M. Palomino-Segura, A. Hidalgo, and F. Diaz-de-Maria, “ACME: Automatic feature extraction for cell migration examination through intravital microscopy imaging,” MEDICAL IMAGE ANALYSIS, vol. 77, 2022.
    [Bibtex] 
    @article{ WOS:000793640300006,
  Author = {Molina-Moreno, Miguel and Gonzalez-Diaz, Ivan and Sicilia, Jon and
  Crainiciuc, Georgiana and Palomino-Segura, Miguel and Hidalgo, Andres
  and Diaz-de-Maria, Fernando},
  Title = {ACME: Automatic feature extraction for cell migration examination
  through intravital microscopy imaging},
  Journal = {MEDICAL IMAGE ANALYSIS},
  Year = {2022},
  Volume = {77},
  Month = {APR},
  DOI = {10.1016/j.media.2022.102358},
  Article-Number = {102358},
  ISSN = {1361-8415},
  EISSN = {1361-8423},
  ResearcherID-Numbers = {Díaz, Iván González/L-5103-2014
  Molina-Moreno, Miguel/ABD-5447-2021
  Palomino-Segura, Miguel/FAE-7067-2022
  },
  ORCID-Numbers = {Díaz, Iván González/0000-0003-4644-8479
  Molina-Moreno, Miguel/0000-0002-3493-7470
  Palomino Segura, Miguel/0000-0003-1614-1222},
  Unique-ID = {WOS:000793640300006},
}
  • [DOI] G. Crainiciuc, M. Palomino-Segura, M. Molina-Moreno, J. Sicilia, D. G. Aragones, J. L. Y. Li, R. Madurga, J. M. Adrover, A. Aroca-Crevillen, S. Martin-Salamanca, A. Serrano del Valle, S. D. Castillo, H. C. E. Welch, O. Soehnlein, M. Graupera, F. Sanchez-Cabo, A. Zarbock, T. E. Smithgall, M. Di Pilato, T. R. Mempel, P. Tharaux, S. F. Gonzalez, A. Ayuso-Sacido, L. G. Ng, G. F. Calvo, I. Gonzalez-Diaz, F. Diaz-de-Maria, and A. Hidalgo, “Behavioural immune landscapes of inflammation,” NATURE, vol. 601, iss. 7893, p. 415+, 2022.
    [Bibtex] 
    @article{ WOS:000739283800002,
  Author = {Crainiciuc, Georgiana and Palomino-Segura, Miguel and Molina-Moreno,
  Miguel and Sicilia, Jon and Aragones, David G. and Li, Jackson Liang Yao
  and Madurga, Rodrigo and Adrover, Jose M. and Aroca-Crevillen, Alejandra
  and Martin-Salamanca, Sandra and Serrano del Valle, Alfonso and
  Castillo, Sandra D. and Welch, Heidi C. E. and Soehnlein, Oliver and
  Graupera, Mariona and Sanchez-Cabo, Fatima and Zarbock, Alexander and
  Smithgall, Thomas E. and Di Pilato, Mauro and Mempel, Thorsten R. and
  Tharaux, Pierre-Louis and Gonzalez, Santiago F. and Ayuso-Sacido, Angel
  and Ng, Lai Guan and Calvo, Gabriel F. and Gonzalez-Diaz, Ivan and
  Diaz-de-Maria, Fernando and Hidalgo, Andres},
  Title = {Behavioural immune landscapes of inflammation},
  Journal = {NATURE},
  Year = {2022},
  Volume = {601},
  Number = {7893},
  Pages = {415+},
  Month = {JAN 20},
  DOI = {10.1038/s41586-021-04263-y},
  EarlyAccessDate = {JAN 2022},
  ISSN = {0028-0836},
  EISSN = {1476-4687},
  ResearcherID-Numbers = {Palomino-Segura, Miguel/FAE-7067-2022
  Tharaux, Pierre-Louis/A-9155-2009
  Díaz, Iván González/L-5103-2014
  Molina-Moreno, Miguel/ABD-5447-2021
  Castillo, Sandra D./AAA-7614-2020
  Sacido, Angel Ayuso/AAV-9331-2021
  Madurga, Rodrigo/ABG-3485-2021
  Adrover, Jose M/G-9741-2017
  Fernandez Calvo, Gabriel/T-6710-2018
  },
  ORCID-Numbers = {Tharaux, Pierre-Louis/0000-0002-6062-5905
  Díaz, Iván González/0000-0003-4644-8479
  Molina-Moreno, Miguel/0000-0002-3493-7470
  Castillo, Sandra D./0000-0002-7007-3155
  Sacido, Angel Ayuso/0000-0003-3919-5880
  Madurga, Rodrigo/0000-0002-1381-2599
  Adrover, Jose M/0000-0002-1395-5477
  Palomino Segura, Miguel/0000-0003-1614-1222
  Fernandez Calvo, Gabriel/0000-0002-3623-236X
  Serrano del Valle, Alfonso/0000-0001-9343-1050
  Aragones, David/0000-0003-4035-0972},
  Unique-ID = {WOS:000739283800002},
}
  • [DOI] A. Gallardo-Antolin and J. M. Montero, “An Auditory Saliency Pooling-Based LSTM Model for Speech Intelligibility Classification,” SYMMETRY-BASEL, vol. 13, iss. 9, 2021.
    [Bibtex] 
    @article{ WOS:000701715100001,
  Author = {Gallardo-Antolin, Ascension and Montero, Juan M.},
  Title = {An Auditory Saliency Pooling-Based LSTM Model for Speech Intelligibility
  Classification},
  Journal = {SYMMETRY-BASEL},
  Year = {2021},
  Volume = {13},
  Number = {9},
  Month = {SEP},
  DOI = {10.3390/sym13091728},
  Article-Number = {1728},
  EISSN = {2073-8994},
  ResearcherID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/L-4152-2014
  Montero, Juan M/K-2381-2014},
  ORCID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/0000-0002-9322-3128
  Montero, Juan M/0000-0002-7908-5400},
  Unique-ID = {WOS:000701715100001},
}
  • [DOI] A. Gallardo-Antolin and J. M. Montero, “Detecting Deception from Gaze and Speech Using a Multimodal Attention LSTM-Based Framework,” APPLIED SCIENCES-BASEL, vol. 11, iss. 14, 2021.
    [Bibtex] 
    @article{ WOS:000678141600001,
  Author = {Gallardo-Antolin, Ascension and Montero, Juan M.},
  Title = {Detecting Deception from Gaze and Speech Using a Multimodal Attention
  LSTM-Based Framework},
  Journal = {APPLIED SCIENCES-BASEL},
  Year = {2021},
  Volume = {11},
  Number = {14},
  Month = {JUL},
  DOI = {10.3390/app11146393},
  Article-Number = {6393},
  EISSN = {2076-3417},
  ResearcherID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/L-4152-2014
  Montero, Juan M/K-2381-2014},
  ORCID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/0000-0002-9322-3128
  Montero, Juan M/0000-0002-7908-5400},
  Unique-ID = {WOS:000678141600001},
}
  • [DOI] A. Gallardo-Antolin and J. M. Montero, “On combining acoustic and modulation spectrograms in an attention LSTM-based system for speech intelligibility level classification,” NEUROCOMPUTING, vol. 456, pp. 49-60, 2021.
    [Bibtex] 
    @article{ WOS:000687471900005,
  Author = {Gallardo-Antolin, Ascension and Montero, Juan M.},
  Title = {On combining acoustic and modulation spectrograms in an attention
  LSTM-based system for speech intelligibility level classification},
  Journal = {NEUROCOMPUTING},
  Year = {2021},
  Volume = {456},
  Pages = {49-60},
  Month = {OCT 7},
  DOI = {10.1016/j.neucom.2021.05.065},
  EarlyAccessDate = {JUN 2021},
  ISSN = {0925-2312},
  EISSN = {1872-8286},
  ResearcherID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/L-4152-2014
  Montero, Juan M/K-2381-2014},
  ORCID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/0000-0002-9322-3128
  Montero, Juan M/0000-0002-7908-5400},
  Unique-ID = {WOS:000687471900005},
}
  • [DOI] J. T. Gomez, A. Rodriguez-Hidalgo, Y. V. J. Naranjo, and C. Pelaez-Moreno, “Teaching Differently: The Digital Signal Processing of Multimedia Content Through the Use of Liberal Arts,” IEEE SIGNAL PROCESSING MAGAZINE, vol. 38, iss. 3, pp. 94-104, 2021.
    [Bibtex] 
    @article{ WOS:000645054800012,
  Author = {Gomez, Jorge Torres and Rodriguez-Hidalgo, Antonio and Naranjo, Yannelys
  Virginia Jerez and Pelaez-Moreno, Carmen},
  Title = {Teaching Differently: The Digital Signal Processing of Multimedia
  Content Through the Use of Liberal Arts},
  Journal = {IEEE SIGNAL PROCESSING MAGAZINE},
  Year = {2021},
  Volume = {38},
  Number = {3},
  Pages = {94-104},
  Month = {MAY},
  DOI = {10.1109/MSP.2021.3053218},
  ISSN = {1053-5888},
  EISSN = {1558-0792},
  ResearcherID-Numbers = {Torres Gómez, Jorge/AAE-8595-2022
  Pelaez-Moreno, Carmen/B-7373-2008},
  ORCID-Numbers = {Torres Gómez, Jorge/0000-0001-9523-048X
  Pelaez-Moreno, Carmen/0000-0003-1425-6763},
  Unique-ID = {WOS:000645054800012},
}
  • [DOI] T. Martinez-Cortes, I. Gonzalez-Diaz, and F. Diaz-de-Maria, “Training deep retrieval models with noisy datasets: Bag exponential loss,” PATTERN RECOGNITION, vol. 112, 2021.
    [Bibtex] 
    @article{ WOS:000615938700018,
  Author = {Martinez-Cortes, Tomas and Gonzalez-Diaz, Ivan and Diaz-de-Maria,
  Fernando},
  Title = {Training deep retrieval models with noisy datasets: Bag exponential loss},
  Journal = {PATTERN RECOGNITION},
  Year = {2021},
  Volume = {112},
  Month = {APR},
  DOI = {10.1016/j.patcog.2020.107811},
  EarlyAccessDate = {JAN 2021},
  Article-Number = {107811},
  ISSN = {0031-3203},
  EISSN = {1873-5142},
  ResearcherID-Numbers = {Díaz, Iván González/L-5103-2014
  Diaz de Maria, Fernando/E-8048-2011},
  ORCID-Numbers = {Díaz, Iván González/0000-0003-4644-8479
  Diaz de Maria, Fernando/0000-0002-6437-914X},
  Unique-ID = {WOS:000615938700018},
}
  • [DOI] F. J. Valverde-Albacete and C. Pelaez-Moreno, “Four-Fold Formal Concept Analysis Based on Complete Idempotent Semifields,” MATHEMATICS, vol. 9, iss. 2, 2021.
    [Bibtex] 
    @article{ WOS:000611373200001,
  Author = {Valverde-Albacete, Francisco Jose and Pelaez-Moreno, Carmen},
  Title = {Four-Fold Formal Concept Analysis Based on Complete Idempotent
  Semifields},
  Journal = {MATHEMATICS},
  Year = {2021},
  Volume = {9},
  Number = {2},
  Month = {JAN},
  DOI = {10.3390/math9020173},
  Article-Number = {173},
  EISSN = {2227-7390},
  ResearcherID-Numbers = {Albacete, Francisco J. Valverde/M-1025-2014
  Pelaez-Moreno, Carmen/B-7373-2008},
  ORCID-Numbers = {Albacete, Francisco J. Valverde/0000-0002-5874-7604
  Pelaez-Moreno, Carmen/0000-0003-1425-6763},
  Unique-ID = {WOS:000611373200001},
}
  • [DOI] M. Fernandez-Diaz and A. Gallardo-Antolin, “An attention Long Short-Term Memory based system for automatic classification of speech intelligibility,” ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, vol. 96, 2020.
    [Bibtex] 
    @article{ WOS:000582708400026,
  Author = {Fernandez-Diaz, Miguel and Gallardo-Antolin, Ascension},
  Title = {An attention Long Short-Term Memory based system for automatic
  classification of speech intelligibility},
  Journal = {ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE},
  Year = {2020},
  Volume = {96},
  Month = {NOV},
  DOI = {10.1016/j.engappai.2020.103976},
  Article-Number = {103976},
  ISSN = {0952-1976},
  EISSN = {1873-6769},
  ResearcherID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/L-4152-2014},
  ORCID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/0000-0002-9322-3128},
  Unique-ID = {WOS:000582708400026},
}
  • [DOI] F. J. Valverde-Albacete and C. Pelaez-Moreno, “The Singular Value Decomposition over Completed Idempotent Semifields,” MATHEMATICS, vol. 8, iss. 9, 2020.
    [Bibtex] 
    @article{ val:pel:20b,
  Author = {Valverde-Albacete, Francisco J. and Pelaez-Moreno, Carmen},
  Title = {The Singular Value Decomposition over Completed Idempotent Semifields},
  Journal = {MATHEMATICS},
  Year = {2020},
  Volume = {8},
  Number = {9},
  Month = {SEP},
  DOI = {10.3390/math8091577},
  Article-Number = {1577},
  EISSN = {2227-7390},
  ResearcherID-Numbers = {Peláez-Moreno, Carmen/B-7373-2008
  Albacete, Francisco J. Valverde/M-1025-2014},
  ORCID-Numbers = {Peláez-Moreno, Carmen/0000-0003-1425-6763
  Albacete, Francisco J. Valverde/0000-0002-5874-7604},
  Unique-ID = {WOS:000580718100001},
}
  • [DOI] A. Vazquez-Romero and A. Gallardo-Antolin, “Automatic Detection of Depression in Speech Using Ensemble Convolutional Neural Networks,” ENTROPY, vol. 22, iss. 6, 2020.
    [Bibtex] 
    @article{ WOS:000553519500001,
  Author = {Vazquez-Romero, Adrian and Gallardo-Antolin, Ascension},
  Title = {Automatic Detection of Depression in Speech Using Ensemble Convolutional
  Neural Networks},
  Journal = {ENTROPY},
  Year = {2020},
  Volume = {22},
  Number = {6},
  Month = {JUN},
  DOI = {10.3390/e22060688},
  Article-Number = {688},
  EISSN = {1099-4300},
  ResearcherID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/L-4152-2014},
  ORCID-Numbers = {GALLARDO-ANTOLÍN, ASCENSIÓN/0000-0002-9322-3128},
  Unique-ID = {WOS:000553519500001},
}
  • [DOI] J. Martinez-Cebrian, M. Fernandez-Torres, and F. Diaz-De-Maria, “Interpretable Global-Local Dynamics for the Prediction of Eye Fixations in Autonomous Driving Scenarios,” IEEE ACCESS, vol. 8, pp. 217068-217085, 2020.
    [Bibtex] 
    @article{ WOS:000600298400001,
  Author = {Martinez-Cebrian, Javier and Fernandez-Torres, Miguel-Angel and
  Diaz-De-Maria, Fernando},
  Title = {Interpretable Global-Local Dynamics for the Prediction of Eye Fixations
  in Autonomous Driving Scenarios},
  Journal = {IEEE ACCESS},
  Year = {2020},
  Volume = {8},
  Pages = {217068-217085},
  DOI = {10.1109/ACCESS.2020.3041606},
  ISSN = {2169-3536},
  ResearcherID-Numbers = {Fernández-Torres, Miguel-Ángel/T-6507-2018
  Diaz de Maria, Fernando/E-8048-2011},
  ORCID-Numbers = {Fernández-Torres, Miguel-Ángel/0000-0002-0801-199X
  Martinez Cebrian, Javier/0000-0002-1882-744X
  Diaz de Maria, Fernando/0000-0002-6437-914X},
  Unique-ID = {WOS:000600298400001},
}
  • [DOI] E. Rituerto-Gonzalez, A. Minguez-Sanchez, A. Gallardo-Antolin, and C. Pelaez-Moreno, “Data Augmentation for Speaker Identification under Stress Conditions to Combat Gender-Based Violence,” APPLIED SCIENCES-BASEL, vol. 9, iss. 11, 2019.
    [Bibtex] 
    @article{rit:min:gal:pel19,
  Author = {Rituerto-Gonzalez, Esther and Minguez-Sanchez, Alba and
  Gallardo-Antolin, Ascension and Pelaez-Moreno, Carmen},
  Title = {Data Augmentation for Speaker Identification under Stress Conditions to
  Combat Gender-Based Violence},
  Journal = {APPLIED SCIENCES-BASEL},
  Year = {2019},
  Volume = {9},
  Number = {11},
  Month = {JUN 1},
  DOI = {10.3390/app9112298},
  Article-Number = {2298},
  EISSN = {2076-3417},
  ResearcherID-Numbers = {Peláez-Moreno, Carmen/B-7373-2008
  Rituerto-González, Esther/AAN-1709-2021
  GALLARDO-ANTOLÍN, ASCENSIÓN/L-4152-2014
  },
  ORCID-Numbers = {Peláez-Moreno, Carmen/0000-0003-1425-6763
  Rituerto-González, Esther/0000-0001-5597-4556
  GALLARDO-ANTOLÍN, ASCENSIÓN/0000-0002-9322-3128
  Minguez Sanchez, Alba/0000-0003-4254-7955},
  Unique-ID = {WOS:000472641200126},
}
  • [DOI] F. J. Valverde-Albacete and C. Pelaez-Moreno, “The Case for Shifting the Renyi Entropy,” ENTROPY, vol. 21, iss. 1, 2019.
    [Bibtex] 
    @article{ val:pel:19c,
  Author = {Valverde-Albacete, Francisco J. and Pelaez-Moreno, Carmen},
  Title = {The Case for Shifting the Renyi Entropy},
  Journal = {ENTROPY},
  Year = {2019},
  Volume = {21},
  Number = {1},
  Month = {JAN},
  DOI = {10.3390/e21010046},
  Article-Number = {46},
  EISSN = {1099-4300},
  ResearcherID-Numbers = {Albacete, Francisco J. Valverde/M-1025-2014
  Peláez-Moreno, Carmen/B-7373-2008},
  ORCID-Numbers = {Albacete, Francisco J. Valverde/0000-0002-5874-7604
  Peláez-Moreno, Carmen/0000-0003-1425-6763},
  Unique-ID = {WOS:000459740300046},
}
  • [DOI] F. J. Valverde-Albacete and C. Peláez-Moreno, “A framework for supervised classification performance analysis with information-theoretic methods,” IEEE Transactions on Knowledge and Data Engineering, pp. 1-1, 2020.
    [Bibtex] 
    @ARTICLE{val:pel:20,
author={F. J. {Valverde-Albacete} and C. {Peláez-Moreno}},
journal={IEEE Transactions on Knowledge and Data Engineering},
title={A framework for supervised classification performance analysis with information-theoretic methods},
year={2020},
volume={},
number={},
pages={1-1},
keywords={Task analysis;Entropy;Mutual information;Proposals;Tools;Performance analysis;Performance evaluation;classification algorithms;information entropy;mutual information;formal concept analysis},
doi={10.1109/TKDE.2019.2915643},
ISSN={2326-3865},
month={},}
  • [DOI] E. Pla-Sacristán, I. González-Díaz, T. Martínez-Cortés, and F. Díaz-de-María, “Finding landmarks within settled areas using hierarchical density-based clustering and meta-data from publicly available images,” Expert Systems with Applications, vol. 123, pp. 315-327, 2019.
    [Bibtex] 
    @article{pla:gon:mar:dia:19,
title = "Finding landmarks within settled areas using hierarchical density-based clustering and meta-data from publicly available images",
journal = "Expert Systems with Applications",
volume = "123",
pages = "315 - 327",
year = "2019",
issn = "0957-4174",
doi = "https://doi.org/10.1016/j.eswa.2019.01.046",
url = "http://www.sciencedirect.com/science/article/pii/S0957417419300521",
author = "Eduardo Pla-Sacristán and Iván González-Díaz and Tomás Martínez-Cortés and Fernando Díaz-de-María",
keywords = "Density-based clustering, K-DBSCAN, V-DBSCAN, Hierarchical clustering, Landmark detection, Tourism",
abstract = "The process of determining relevant landmarks within a certain region is a challenging task, mainly due to its subjective nature. Many of the current lines of work include the use of density-based clustering algorithms as the base tool for such a task, as they permit the generation of clusters of different shapes and sizes. However, there are still important challenges, such as the variability in scale and density. In this paper, we present two novel density-based clustering algorithms that can be applied to solve this: K-DBSCAN, a clustering algorithm based on Gaussian Kernels used to detect individual inhabited cores within regions; and V-DBSCAN, a hierarchical algorithm suitable for sample spaces with variable density, which is used to attempt the discovery of relevant landmarks in cities or regions. The obtained results are outstanding, since the system properly identifies most of the main touristic attractions within a certain region under analysis. A comparison with respect to the state-of-the-art show that the presented method clearly outperforms the current methods devoted to solve this problem."
}
  • [DOI] M. Fernández-Torres, I. González-Díaz, and F. Díaz-de-María, “Probabilistic Topic Model for Context-Driven Visual Attention Understanding,” IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2019.
    [Bibtex] 
    @ARTICLE{tor:gon:dia:19,
author={M. {Fernández-Torres} and I. {González-Díaz} and F. {Díaz-de-María}},
journal={IEEE Transactions on Circuits and Systems for Video Technology},
title={Probabilistic Topic Model for Context-Driven Visual Attention Understanding},
year={2019},
volume={},
number={},
pages={1-1},
keywords={Visualization;Task analysis;Adaptation models;Feature extraction;Computational modeling;Probabilistic logic;Context modeling;Top-down visual attention;hierarchical probabilistic framework;context-aware model;latent topic models},
doi={10.1109/TCSVT.2019.2909427},
ISSN={1558-2205},
month={},}
  • [DOI] M. Molina-Moreno, I. González-Díaz, and F. Díaz-de-María, “Efficient Scale-Adaptive License Plate Detection System,” IEEE Transactions on Intelligent Transportation Systems, pp. 1-13, 2018.
    [Bibtex] 
    @ARTICLE{Molina-Moreno2018, 
author={M. Molina-Moreno and I. González-Díaz and F. Díaz-de-María}, 
journal={IEEE Transactions on Intelligent Transportation Systems}, 
title={Efficient Scale-Adaptive License Plate Detection System}, 
year={2018}, 
volume={}, 
number={}, 
pages={1-13}, 
keywords={Licenses;Detectors;Feature extraction;Deformable models;Lighting;Robustness;Image edge detection;License plate detection;GentleBoost;scale-adaptive part-based model;video surveillance}, 
doi={10.1109/TITS.2018.2859035}, 
ISSN={1524-9050}, 
month={},}
  • [DOI] F. Fernández-Martínez, A. Hernández-García, M. A. Fernández-Torres, I. González-Díaz, Á. García-Faura, and F. Díaz-de-María, “Exploiting visual saliency for assessing the impact of car commercials upon viewers,” Multimedia Tools and Applications, vol. 77, iss. 15, pp. 18903-18933, 2018.
    [Bibtex] 
    @ARTICLE{FernandezMartinez2018, 
author="Fern{\'a}ndez-Mart{\'i}nez, F.
and Hern{\'a}ndez-Garc{\'i}a, A.
and Fern{\'a}ndez-Torres, M. A.
and Gonz{\'a}lez-D{\'i}az, I.
and Garc{\'i}a-Faura, {\'A}.
and  D{\'i}az-de-Mar{\'i}a, F.",
journal={Multimedia Tools and Applications}, 
title={Exploiting visual saliency for assessing the impact of car commercials upon viewers}, 
year={2018}, 
volume={77}, 
number={15}, 
pages={18903-18933}, 
keywords={Visual attention, Saliency, Scene analysis, Aesthetics assessment, Feature extraction, Video impact assessment}, 
doi={10.1007/s11042-017-4879-3}, 
ISSN={1573-7721}, 
month={August},}
  • [DOI] F. J. Valverde-Albacete and C. Peláez-Moreno, “The Case for Shifting the Rényi Entropy,” Entropy, vol. 21, iss. 1, 2019.
    [Bibtex] 
    @Article{val:pel:19,
AUTHOR = {Valverde-Albacete, Francisco J. and Peláez-Moreno, Carmen},
TITLE = {The Case for Shifting the Rényi Entropy},
JOURNAL = {Entropy},
VOLUME = {21},
YEAR = {2019},
NUMBER = {1},
ARTICLE-NUMBER = {46},
URL = {http://www.mdpi.com/1099-4300/21/1/46},
ISSN = {1099-4300},
ABSTRACT = {We introduce a variant of the R\´enyi entropy definition that aligns it with the well-known H\"older mean: in the new formulation, the r-th order Renyi Entropy is the logarithm of the inverse of the r-th order H\"older mean. This brings about new insights into the relationship of the R\´enyi entropy to quantities close to it, like the information potential and the partition function of statistical mechanics. We also provide expressions that allow us to calculate the R\' enyi entropies from the Shannon cross-entropy and the escort probabilities. Finally, we discuss why shifting the R\`enyi entropy is fruitful in some applications.},
DOI = {10.3390/e21010046}
}
  • [DOI] F. J. Valverde-Albacete and C. Peláez-Moreno, “K-Formal Concept Analysis as linear algebra over idempotent semifields,” Information Sciences, vol. 467, pp. 579-603, 2018.
    [Bibtex] 
    @article{val:pel:18c,
title = "K-Formal Concept Analysis as linear algebra over idempotent semifields",
journal = "Information Sciences",
volume = "467",
pages = "579 - 603",
year = "2018",
issn = "0020-0255",
doi = "https://doi.org/10.1016/j.ins.2018.07.067",
url = "http://www.sciencedirect.com/science/article/pii/S0020025516312051",
author = "Francisco J. Valverde-Albacete and Carmen Pel\'aez-Moreno",
keywords = "Generalised Formal Concept Analysis, Concept lattice, Neighborhood lattice, Idempotent semiring, Dioid, Confusion matrix",
abstract = "We report on progress in characterizing K-valued FCA in algebraic terms, where K is an idempotent semifield. In this data mining-inspired approach, incidences are matrices and sets of objects and attributes are vectors. The algebraization allows us to write matrix-calculus formulae describing the polars and the fixpoint equations for extents and intents. Adopting also the point of view of the theory of linear operators between vector spaces we explore the similarities and differences of the idempotent semimodules of extents and intents with the subspaces related to a linear operator in standard algebra. This allows us to shed some light into Formal Concept Analysis from the point of view of the theory of linear operators over idempotent semimodules. In the opposite direction, we state the importance of FCA-related concepts for dual order homomorphisms of linear spaces over idempotent semifields, specially congruences, the lattices of extents, intents and formal concepts."
}
  • [DOI] A. Rodríguez-Hidalgo, C. Peláez-Moreno, and A. Gallardo-Antolín, “The Robustness of Echoic Log-Surprise Auditory Saliency Detection,” IEEE Access, vol. 6, pp. 72083-72093, 2018.
    [Bibtex] 
    @ARTICLE{rod:pel:gal:18b,
author={A. Rodr\'iguez-Hidalgo and C. Pel\'aez-Moreno and A. Gallardo-Antol\'in},
journal={IEEE Access},
title={The Robustness of Echoic Log-Surprise Auditory Saliency Detection},
year={2018},
volume={6},
number={},
pages={72083-72093},
keywords={Acoustics;Robustness;Task analysis;Saliency detection;Signal processing algorithms;Bayes methods;Spectrogram;Acoustic saliency;echoic memory;multi-scale;statistical divergence;Jensen-Shannon;acoustic event detection},
doi={10.1109/ACCESS.2018.2882055},
ISSN={2169-3536},
month={},}

Conferences

  • [DOI] F. J. Valverde-Albacete, C. Pelaez-Moreno, I. P. Cabrera, P. Cordero, and M. Ojeda-Aciego, “Formal Independence Analysis,” in INFORMATION PROCESSING AND MANAGEMENT OF UNCERTAINTY IN KNOWLEDGE-BASED SYSTEMS: THEORY AND FOUNDATIONS, IPMU 2018, PT I, 2018, pp. 596-608.
    [Bibtex] 
    @inproceedings{ WOS:000481659500051,
  Author = {Valverde-Albacete, Francisco J. and Pelaez-Moreno, Carmen and Cabrera,
  Inma P. and Cordero, Pablo and Ojeda-Aciego, Manuel},
  Editor = {Medina, J and OjedaAciego, M and Verdegay, JL and Pelta, DA and Cabrera, IP and BouchonMeunier, B and Yager, RR},
  Title = {Formal Independence Analysis},
  Booktitle = {INFORMATION PROCESSING AND MANAGEMENT OF UNCERTAINTY IN KNOWLEDGE-BASED
  SYSTEMS: THEORY AND FOUNDATIONS, IPMU 2018, PT I},
  Series = {Communications in Computer and Information Science},
  Year = {2018},
  Volume = {853},
  Number = {I},
  Pages = {596-608},
  Note = {17th International Conference on Information Processing and Management
  of Uncertainty in Knowledge-Based Systems (IPMU), Cadiz, SPAIN, JUN
  11-15, 2018},
  DOI = {10.1007/978-3-319-91473-2\_51},
  ISSN = {1865-0929},
  EISSN = {1865-0937},
  ISBN = {978-3-319-91473-2; 978-3-319-91472-5},
  ResearcherID-Numbers = {Peláez-Moreno, Carmen/B-7373-2008
  Ojeda-Aciego, Manuel/E-7617-2012
  Valverde Albacete, Francisco Jose/M-1025-2014},
  ORCID-Numbers = {Peláez-Moreno, Carmen/0000-0003-1425-6763
  Ojeda-Aciego, Manuel/0000-0002-6064-6984
  Penas Cabrera, Inmaculada de las/0000-0001-5129-0085
  Valverde Albacete, Francisco Jose/0000-0002-5874-7604},
  Unique-ID = {WOS:000481659500051},
}
  • [DOI] F. J. V. Albacete, C. Peláez-Moreno, P. Cordero, and M. Ojeda-Aciego, “Formal Equivalence Analysis,” in 2019 Conference of the International Fuzzy Systems Association and the European Society for Fuzzy Logic and Technology (EUSFLAT 2019), 2019.
    [Bibtex] 
    @inproceedings{val:pel:cor:oje:19,
  title={Formal Equivalence Analysis},
  author={Francisco José Valverde Albacete and Carmen Peláez-Moreno and Pablo Cordero and Manuel Ojeda-Aciego},
  year={2019},
  booktitle={2019 Conference of the International Fuzzy Systems Association and the European Society for Fuzzy Logic and Technology (EUSFLAT 2019)},
  issn={2589-6644},
  isbn={978-94-6252-770-6},
  url={https://doi.org/10.2991/eusflat-19.2019.109},
  doi={https://doi.org/10.2991/eusflat-19.2019.109},
  publisher={Atlantis Press}
}
  • [DOI] A. Gallardo-Antolín and J. M. Montero, “A Saliency-Based Attention LSTM Model for Cognitive Load Classification from Speech,” in Proc. Interspeech 2019, 2019, pp. 216-220.
    [Bibtex] 
    @inproceedings{gal:mon:19,
  author={Ascensión Gallardo-Antolín and Juan Manuel Montero},
  title={{A Saliency-Based Attention LSTM Model for Cognitive Load Classification from Speech}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={216--220},
  doi={10.21437/Interspeech.2019-1603},
  url={http://dx.doi.org/10.21437/Interspeech.2019-1603}
}
  • [DOI] T. Martínez-Cortés, I. González-Díaz, and F. Díaz-de-María, “Automatic Learning of Image Representations Combining Content and Metadata,” in 2018 25th IEEE International Conference on Image Processing (ICIP), 2018, pp. 1972-1976.
    [Bibtex] 
    @INPROCEEDINGS{Martinez-Cortes2018, 
author={T. Martínez-Cortés and I. González-Díaz and F. Díaz-de-María}, 
booktitle={2018 25th IEEE International Conference on Image Processing (ICIP)}, 
title={Automatic Learning of Image Representations Combining Content and Metadata}, 
year={2018}, 
volume={}, 
number={}, 
pages={1972-1976}, 
keywords={feedforward neural nets;image representation;learning (artificial intelligence);meta data;automatic training framework;image visual contents;image descriptors;location-related information;automatic learning;image representation;deep convolutional neural networks;image visual content;metadata;landmark discovery task;content-based image representation;visual-related information;loss-function;Visualization;Metadata;Task analysis;Training;Urban areas;Global Positioning System;Computational modeling;CNN;metadata;loss function;weak labels}, 
doi={10.1109/ICIP.2018.8451566}, 
ISSN={2381-8549}, 
month={Oct},}
  • I. González-Díaz, J. Benois-Pineau, J. Domenger, and A. de Rugy, “Perceptually-guided Understanding of Egocentric Video Content: Recognition of Objects to Grasp,” in ACM International Conference on Multimedia Retrieval, ICMR, 2018, pp. 434-441.
    [Bibtex] 
    @inproceedings{Gonzalez2018,
  author    = {Iv{\'{a}}n Gonz{\'{a}}lez{-}D{\'{i}}az and
               Jenny Benois{-}Pineau and
               Jean{-}Philippe Domenger and
               Aymar de Rugy},
  title     = {Perceptually-guided Understanding of Egocentric Video Content: Recognition
               of Objects to Grasp},
  booktitle = {ACM International Conference on Multimedia Retrieval, {ICMR}},
  pages     = {434--441},
  year      = {2018},
  timestamp = {Mon, 11 Jun 2018 09:27:11 +0200},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
  • F. J. Valverde-Albacete, C. Peláez-Moreno, I. P. Cabrera, P. Cordero, and M. Ojeda-Aciego, “A Data Analysis Application of Formal Independence Analysis,” in Concept Lattices and their Applications (CLA 2018), , 2018, pp. 1-12.
    [Bibtex] 
    @incollection{val:pel:cab:cor:oje:18b,
  Author = {Valverde-Albacete, Francisco J and Pel{\'a}ez-Moreno, Carmen and Cabrera, Inma P and Cordero, P and Ojeda-Aciego, Manuel},
  Booktitle = {Concept Lattices and their Applications (CLA 2018)},
  Date-Added = {2018-05-08 07:39:20 +0000},
  Date-Modified = {2018-05-08 07:39:20 +0000},
  Pages = {1--12},
  Title = {{A Data Analysis Application of Formal Independence Analysis}},
  Year = {2018}}

PhDThesis

  • M. Á. Fernández-Torres, “Hierarchical representations for spatio-temporal visual attention modeling and understanding,” PhD Thesis, 2019.
    [Bibtex] 
    @phdthesis{tesis6,
  author       = {Miguel Ángel Fernández-Torres}, 
  title        = {Hierarchical representations for spatio-temporal visual attention modeling and understanding},
  school       = {Escuela Politécnica Superior, Universidad Carlos III de Madrid.},
  year         = 2019,
  month        = 2,
  note         = {An optional note}
}
  • A. Rodríguez-Hidalgo, “Bayesian and Echoic Log-surprise for auditory saliency detection,” PhD Thesis, 2019.
    [Bibtex] 
    @phdthesis{tesis7,
  author       = {Antonio Rodríguez-Hidalgo}, 
  title        = {Bayesian and Echoic Log-surprise for auditory saliency detection},
  school       = {Escuela Politécnica Superior, Universidad Carlos III de Madrid.},
  year         = 2019,
  month        = 2,
  note         = {An optional note}
}
  • J. López-Labraca, “Contributions to Melanoma Computer Aided Diagnosis Systems using Dermoscopic Images,” PhD Thesis, 2020.
    [Bibtex] 
    @phdthesis{tesis8,
  author       = {Javier López-Labraca}, 
  title        = {Contributions to Melanoma Computer Aided Diagnosis Systems using Dermoscopic Images},
  school       = {Escuela Politécnica Superior, Universidad Carlos III de Madrid.},
  year         = 2020,
  month        = 1,
  note         = {An optional note}
}
  • T. Martínez-Cortés, “Training deep retrieval models with noisy datasets,” PhD Thesis, 2021.
    [Bibtex] 
    @phdthesis{tesis9,
  author       = {Tomás Martínez-Cortés}, 
  title        = {Training deep retrieval models with noisy datasets},
  school       = {Escuela Politécnica Superior, Universidad Carlos III de Madrid.},
  year         = 2021,
  month        = 3,
  note         = {An optional note}
}
  • E. P. Sacristán, “Density-based clustering: algorithms and evaluation techniques,” PhD Thesis, 2021.
    [Bibtex] 
    @phdthesis{tesis10,
  author       = {Eduardo Pla Sacristán}, 
  title        = {Density-based clustering: algorithms and evaluation techniques},
  school       = {Escuela Politécnica Superior, Universidad Carlos III de Madrid.},
  year         = 2021,
  month        = 29
  note         = {An optional note}
}

Fechas de ejecución: 01/01/2018-31/09/2021

Financiado por: FEDER/Ministerio de Ciencia, Innovación y Universidades – Agencia Estatal de Investigación/TEC2017-84395-P

Comments are closed.