Reenable multiple elasticsearch indices
In https:/
To reiterate the reason, a number of plugins use has_parent queries in RBAC filters. The searchlight query is run across all appropriate indices with a series of `indices` filters, one per resource type, that contain the filters for that type. When a type does not exist at all in an index, has_parent (or has_child) queries will fail, and unfortunately the query parser doesn't take into account that a particular filter should not be run on a particular index.
I'm hoping for a response to https:/
* Use a multi-search (https:/
* Create (empty?) type mappings across all indices (though possibly include type mappings). This actually seems like an OK solution. If there's a way to ensure no documents are inappropriately indexed, that would be excellent.
This may not need a full spec document since it's a straightforward idea.
After some consideration, we will elect to create empty mappings across all indices. This will be done once at startup. After all of the "real" mappings are created we will cycle through all indices. For each index we will cycle through each doc_type. If a mapping does not exist for that doc_type, we will create it. We should add a single bogus field that would never be used and mark the mapping as "static". This should prevent it from every being used.
Here is a quick bash script using curl to show the concept:
# We will place the following doc_types in Index 1:
# OS::Swift::Account, OS::Swift:
# OS::Swift::Object [Child of Container], OS::Nova::Server
# We will place the following doc_types in Index 2:
# OS::Designate:
# OS::Cinder::Volume, OS::Cinder:
TYPES="
INDICES="index1 index2"
HOST=localhost
PORT=9200
IP="http://
# Clean out Elasticsearch
curl -s -XDELETE "${IP}/_all" >/dev/null
# Set up Index1
curl -s -XPUT "${IP}/index1" -d'{"mappings": {"OS::Swift:
curl -s -XPUT "${IP}/
curl -s -XPUT "${IP}/
curl -s -XPUT "${IP}/
# Do a query on Index1 based on an Index1 doc_type: Pass
echo "Query Passes"
curl -s -XGET "${IP}/
echo ""
# Set up Index2
curl -s -XPUT "${IP}/index2" -d'{"mappings": {"OS::Designate
curl -s -XPUT "${IP}/
curl -s -XPUT "${IP}/
curl -s -XPUT "${IP}/
# Do a query on Index1 based on an Index1 doc_type: Pass
echo "Query Passes"
curl -s -XGET "${IP}/
echo ""
# Do a query on Index2 based on an Index2 doc_type: Pass
echo "Query Passes"
curl -s -XGET "${IP}/
echo ""
# Do a query across both indices based on an index1 doc_type: Fail
echo "Query Fails"
curl -s -XGET "${IP}/
echo ""
# Do a query across both indices based on an index2 doc_type: Fail
echo "Query Fails"
curl -s -XGET "${IP}/
echo ""
# Proof Of Concept: Fill out indcies with "missing" doc-types.
for ndx in ${INDICES}
do
for type in ${TYPES}
do
if [ "${mapping}" = "{}" ]
then
curl -s -XPUT "${IP}/
echo ""
fi
done
done
# Do a query across both indices based on an index1 doc_type: Pass!
echo "Query Passes"
curl -s -XGET "${IP}/
echo ""
# Do a query across both indices based on an index2 doc_type: Pass!
echo "Query Passes"
curl -s -XGET "${IP}/
echo ""
Blueprint information
- Status:
- Complete
- Approver:
- Travis Tripp
- Priority:
- High
- Drafter:
- Steve McLellan
- Direction:
- Approved
- Assignee:
- Rick Aulino
- Definition:
- Approved
- Series goal:
- Accepted for newton
- Implementation:
-
Implemented
- Milestone target:
-
newton-1
- Started by
- Travis Tripp
- Completed by
- Travis Tripp
Related branches
Related bugs
Sprints
Whiteboard
Gerrit topic: https:/
Addressed by: https:/
Reenable multiple elasticsearch indices
Gerrit topic: https:/