In this document, we explain relevance rating guidelines for video search on Apple TV. If you are not familiar with the Apple TV app, please refer to https://www.apple.com/apple-tv-app/ for an overview and basic information about this app.
1.1 The importance of your work as a Rater
Each of the judgements you complete will be used to build and improve artificial intelligence systems such as search algorithms and machine learned rankers that power the user experience for Apple TV users. Your attention to detail, research and language skills as well as your cultural knowledge of the market are all critical to the success of our projects.
Your judgements should represent those of an Apple TV user who is using the Search feature. Ask yourself if you would be content with the results returned for a particular search query. Is there a significant relationship between the query and content returned? Would you be content if you see this content appear as a search result? Stay curious and complete thorough research.
Our ultimate goal is to surprise and delight our customers by improving search quality and enhancing customer satisfaction, and you play an important role in this.
Please keep in mind that your tasks will be spot-checked for quality, and measured against those of your peers.
1.2 Primary and Secondary Intent
The primary intent of a query is the most likely intent, i.e. the intent of most users who say the given query. Some queries may have multiple primary intents.
A secondary intent is less likely, or would be a less popular intent compared to a primary one. A secondary intent could be:
Content relevant to a smaller group of users than for the primary intent. For queries like [shows] and [movies] the primary intent is usually media content for grown-ups. Content for children would be considered secondary intent, except if the intent is obviously kids-related (such as cartoons, animated films, etc.).
Complimentary content such as trailers, reviews, cast members, or interviews with the cast on how the movie was made.
Lower quality/lesser known content that is relevant to the query but is not the primary intent, a content that is dated or less popular.
1.3 Query Types & Intents
This evaluation contains long, complex queries that may contain multiple aspects, such as specifying a genre, time-period and actor all in one query. These queries will broadly fall into three different categories that will be rated differently.
1.3.1 Browse
The first type is queries with a browsing intent. These queries will point to a larger set of content where the user doesn’t have anything specific in mind. Some examples:
[best tv shows set in the future]
[i want a a classic made for tv type move]
[show me commedies available in french]
[whats something good to watch around chinese new years]
1.3.2 Navigational
The second type of queries have a navigational intent. These queries are looking for a specific piece of content or a small list of contents. Some examples:
[most recent best picture academy awards] → Looking for a specific movie
[james bond movies with pierce brosnan] → Pierce Brosnan was James Bond in 4 films
[that tv+ show where the actor travels the world even though he hates travel] → This describes The Reluctant Traveler with Eugene Levy
[that bond movie quantum something] → Looking for Quantum of Solace
1.3.3 Similarity
The final type of query is when the user is looking for content similar to another piece of content. These queries will generally reference a piece(s) of content and use phrases like “similar to” or “like”. Some examples:
[movies like top gun]
[suspensful show similar to the 100]
[show me something super dark like sey7en]
1.4 Reasoning
For these complex queries it is especially important to provide the reasoning for a result being relevant to the query. Thus, for each result you must provide the following in the “Reasoning” field.
Provide a concise explanation indicating why it should be considered relevant, not relevant, or the relevance considered as ambiguous via unknown. Your explanation MUST answer each of the following questions:
What is the intent of the query?
Are there any abbreviations? If so, explain them (e.g. "90s refers to the time between 1990 and 1999").
How is the result relevant to the query?
The reasoning field should be primarily focused on the connection of the result to the query rather than explaining the specific rating given.
1.4.1 Reasoning Examples
Below are examples of the reasoning for different query and result pairs. The examples are color coded to show how the questions are answered. reasoning examples
2 Rating Process
2.1 Query Research
To complete query research, tap into yourlocal market knowledge in addition toonline sourcessuch as IMDB, YouTube, Wikipedia, video streaming services, local content evaluators and social media. Consider how a user of Apple TV engages the search feature to navigate to a specific set of content, or as a means to browse a larger catalogue of titles.
Online Research
IMDB
Popularity Ranking
Rating Counts
Storyline, Taglines
Genre
Release Date
For topic, decade, etc: (IMDB Sort By feature)
Box Office Mojo
Local Box Office Rankings
Common Sense Media
Age Rating → for determination of kids content
Wikipedia
Information of the movie/show: plot, cast, awards, nominations, crew
Social Media
Trending films
Followers on Twitter, Instagram, TikTok, etc. can help to determine a person’s popularity
YouTube trailer views
Make sure to factor in time, consider the monthly average views since its upload
Local content evaluators
To identify popularity in the market
2.2 Aspects to Consider
2.2.1 Connection to Query
The results that users see when searching can broadly be categorized into two groups: Relevance and Similarity. Below are examples where the show Ted Lasso might surface due to Relevance or Similarity. In addition there is a short, non-comprehensive, list of other relevant content.
2.2.1.1 Relevance
[the show with the american coach in europe with the mustache] → While indirect, this query is looking for a show with an American Coach that has a mustache and is in Europe, this accurately describes the titular character of Ted Lasso
Plot point is specific to Ted Lasso
[commedy tv shows to binge] → Despite being misspelled this query is clearly looking for comedy tv shows, which Ted Lasso is
The Office, Parks and Rec, South Park
[emmy award winners since 2020] → Ted Lasso and its cast and crew have won multiple Emmy’s since 2020
Succession, Last Week Tonight with John Oliver, The Bear
[best apple originals that are funny] → The show is produced by and streamed on Apple TV+ and is a comedy
Shrinking, Loot, Trying
[I want some shows about soccer clubs] → Ted Lasso is about a soccer team
Welcome to Wrexham, Boca Juniors Confidential, All or Nothing: Tottenham Hotspur
[what are some of the best shows with jason sudeikis] → The show stars Jason Sudeikis
Saturday Night Live, 30 Rock, The Last Man on Earth
[I want to watch something where a protege becomes a rival] → This query is describing a key plot point of season 3 of Ted Lasso
Star Wars (Anakin & Obi Wan), Naruto (Orochimaru & 3rd Hokage)
2.2.1.2 Similarity
[sports dramas like firday night lights] → This query is looking for something similar to Friday Night Lights which is a sports drama. Ted Lasso is also a sports drama and focuses around a charismatic head coach, their age ratings (TV-14 and TV-MA are close) making them very similar and Ted Lasso Relevant
All American, The Boys in the Boat
[apple tv+ shows similar to shrinking] → Ted Lasso is an Apple TV+ dramatic comedy with a TV-MA age rating, just like Shrinking, making it quite similar
Loot, Platonic, Trying
2.2.1.3 Mispelling/Alternate spelling
Some words can be spelled in multiple ways or frequently misspelled. Since search is able to handle misspellings, please evaluate the intent rather than the exact text.
[rhe] / [thr ] / [yhe] → Common misspellings for the word “the”
[nruto] → Likely looking for “Naruto”
[bestmovies] → Probably intended as “best movies”
[4], [IV], [Four] → consider synonyms
2.2.1.4 Entity Type Filters
Some queries may include keywords such as “movies”, “shows”, “documentaries”. For these queries anything content that does not fit that entity type is considered not relevant and off-topic.
[best korean romance shows] → movies are not relevant
[documentaries about insects] → only documentaries are relevant, A Bug’s Life would be off-topic
2.2.1.5 Partial Relevance
Results may match on part of a query, in these instances it is important to understand how well the full intent is satisfied. We can think of the various requirements of a query existing on a scale from Factual to Ambiguous and how we treat a partial query intent match depends on this.
For fully Factual aspects anything that doesn’t match would be considered off-topic, while for more Ambiguous requirements we would demote the rating by 1 (i.e. Good → Acceptable) if the result matches the other aspects but not that requirement.
An example of how this comes into play would be with the query [exciting 2000’s documentary with morgan freeman]
Exciting is a mood and hard to define, if the result is a 2000’s documentary with Morgan Freeman that you as a rater don’t consider exciting it still likely fully satisfies the intent as we can’t really define what an “exciting documentary is”
Note that for a query like [exciting dramas with morgan freeman] it would be more reasonable to take a look at the plot and determine if the movie appears exciting, not fulfilling this requirement would lead to a demotion of 1 rating if we returned a “boring” drama with Morgan Freeman.
2000’s is quite factual, the content needs to be from the years 2000 to 2009 or explicitly about that time period to be considered fully relevant. If the content is from prior to that time period it would be off-topic. If the content is from 2010 onwards it would still have some relevance as “2000’s” doesn’t explicitly exclude the later years in that way that “90’s” and other decades queries do. In this case a documentary from 2020 would get a demotion of 1 rating.
Documentary is also quite factual, but not a hard requirement. We might have a returned result that is an exciting 2000’s movie with morgan freeman that is explicitly based on a true story. While this isn’t an actual documentary it could be considered partially fulfilling the intent and relevant with a demotion of 1 rating.
Morgan Freeman is a fully factual requirement. Any content without Morgan Freeman would be considered off-topic.
Note: The above guidance is primarily for Browse queries, on a Navigational query consider the above if the result is a secondary intent, but otherwise apart from the query intent the results will likely be judged on similarity.
2.2.2 Popularity
Popularity is a key component of ratings, sources such as IMDB, YouTube trailer views, Box Office returns will help determine how popular a piece of content is.
Remember that popularity is a sliding scale depending on the relevant content for a query. A query such as [action comedies that don’t have sci-fi elements] has a lot more relevant content than [norwegian comedy shows about school children]. For our ratings there might be a Norwegian action comedy show that is considered very popular for the second query and not popular for the first because of the different set of relevant results.
2.2.2.1 AppleTV+ Content
Users are using search on Apple TV which indicates a preference for ATV+ content such as Ted Lasso, Coda, Severance or Tehran. Therefore
Popularity is slightly less strict for ATV+ content
If you are in doubt about “Good” or “Excellent” for an ATV+ result, select “Excellent”
2.2.2.2 Pre-Release Content
Sometimes the result is the preview page for content which is not yet available for viewing. The timing of the release can be ignored. Different sources will need to be utilized to determine popularity than for already released content.
Complete research to determine expected popularity (ex. YouTube trailers, production budget, famous cast and crew)
Original content from ATV+, Netflix, Hulu, or other popular production company - generally assumed to be popular.
2.2.2.3 Seasonal Results
In some locales there are seasonal results which have additional relevance and popularity at specific times of year, for example holiday movies are very popular in the US at the end of the year. Please consider the seasonal popularity when rating results and consider increasing the rating due to seasonal popularity.
Examples:
[s] → “Spirited”: Always “excellent”, new and very popular result, excellent is the best result for an “Ambiguous - Intent Unclear” query
[c] → “A Charlie Brown Christmas”: If rating the result in July this would be a “good” result as it is relevant and popular, but not quite at the level of a classic that would receive an excellent rating. However, if this result surfaces in the November / December timeframe it should be rated as excellent as it is a holiday classic and an excellent result at that time
[bo] → “A Boyfriend for Christmas”: If rating in the summer this would be an “acceptable” result, it is relevant but not recent or popular. During the Holiday season it would be rated as "good", being a Christmas Romantic Comedy, it has additional popularity during that period
2.2.3 Recency / Time Period
In general users are more satisfied with newer content and our ratings should prioritize content that has not been out as long. As with popularity, recency exists on a sliding scale, with broad queries such as [what's something I can watch with surround sound] having more relevant results than [historical dramas where they speak latin].
For TV Shows recency is based on the most recent season.
2.2.3.1 Time Period Queries
Some queries will specify a time period, i.e. [action movies from the 80’s]. In this case recency does not matter, instead it is whether the content fits the queried time frame that matters. For those queries focus on popularity in particular for the final rating. Note: TV shows can be relevant over many years / multiple time periods based on when their seasons aired.
[i want to watch a classic 90’s sitcom] / [2000’s most popular shows] Friends is relevant for both of these queries
Please refer to 3.4.1 Time Period for more details and examples for rating
2.2.3.2 Classic Content
Ignore recency for classic movies and tv shows that were among the most popular in the decade they were released or have high popularity even today.
2.2.4 Other Rating Modifiers
2.2.4.1 Kids Content
As kids content satisfies the requests for a subset of users, it shall not get the same ratings for generic queries as content for all age groups. Therefore, kids content shall be demoted by one ratinglevel unless the query is regarding kids content either directly (navigational) or indirectly (“kids” token in query, animation, “for the family”, etc.). Please make use of the “Kids&Family” genre, Common Sense Media and your own knowledge to determine whether content is targeting kids specifically. Additionally, do not demote relevant content from Acceptable → Off-Topic as relevant content hsould always be rated as at least acceptable. Examples:
[adventure shows to binge this weekend] → “Yakari”: The returned show is in the intended genre and very popular, however dated and hence could be rated as good. However, since it is primarily kids content, it will be rated acceptable.
2.2.4.2 Franchise Ratings
General Guideline:
Perfect → The first and last item of core franchise. Please make an honest determination of the order in which users would typically watch the collection or franchise of content and use this to complete the rating. Use comment to justify decision.
Excellent → Other content in core franchise
content in the franchise, but not core
Excellent → popular and recent
Good → popular or recent
Acceptable → neither popular nor recent
2.2.4.3 Result filters
You may come across queries that include the tokens [“new”, “best”]. Additionally to the core intent, these function like filters. When this is a part of the query, please consider the implications for your rating:
[new] → emphasize on recency aspect, the popularity aspect is less important
[best] → emphasize on the popularity aspect, the recency aspect is less important
2.2.4.4 Movie Bundles
Movie bundles shall be rated for the content included that is related to the query. Once the rating for the content is determined, demote rating by one level
If the movie bundle contains the primary intended content it can receive an Excellent rating
Example: No time to Die → James Bond 10 Film collection → Excellent
If the movie bundle contains a piece of content that individually would be rated as “Acceptable”, the bundle receives the “Unacceptable: Off-Topic” rating
2.2.4.5 Apple Event / WWDC
If this is the primary intent (query = “Apple Event replays”, “wwdc with a new iphone announced”, “apple developer conference film”), rate based on recency. Rating score decreases per year until “Acceptable”. Year is considered the last 12 months, so the past year on October 31, 2022 would be any events from November 2021 through the current date.
If the Apple Event / WWDC content is relevant to the query but unlikely to be the intent, rate as “Acceptable”.
Query = “2022”, “w”, “September 2021”
*date of publishing these examples: 3/7/24
2.2.4.6 Explicit Content
Rate all pornographic content as Unacceptable: Off-Topic. Exception: If the query intent is specifically for erotic content, or porn → Please use above guidelines to find appropriate rating.
[drama movies for me and my friends] → any pornographic content is Unacceptable: Off-Topic
[erotica] / [porn] → erotic content can receive any rating
[18+] / [movies for adults] → Intent here is not specifically for pornographic or erotic content
2.2.4.7 Free Queries
For queries with strings such as “free”, “free for me”, please consider what streaming services in your market are free along with content that is available explicitly for free. This includes TV shows where the first episode is available for free. Use the below guidance to rate these queries.
Short queries that include the string “free” (“free movies”, “free shows”, “free”)
Excellent → all AppleTV+ originals
Good → non AppleTV+ content that is popular and recent
Acceptable → non AppleTV+ content that is popular or recent
Off-Topic → non Apple TV+ content that is neither popular, nor recent
Longer queries that include the string “free” (“free comedy movies”, “free horror shows”, “scary movies free for me”)
Check if the content is free and then rate based on other aspects of content
“free denzel 2000s”
check if content is free, from the 2000s and features Denzel Washington
if yes and Denzel Washington stars in the content then rating is excellent
If Denzel is in a lesser role, rate based on recency and popularity
“free comedy movies”
check if content is free, a comedy and a movie
if yes then rate based on recency and popularity
Note: In addition to using traditional web resources, one can also check if a content is free follow these steps:
Click the link that is the title of the returned contentproduct page link
This will bring you to apple tv product page of the content in the country of the evaluation
Availability
Here you will see what platforms the content is available through
For instance in the US free streaming platforms include (but are not limited too): Tubi TV, Pluto TV, Amazon FreeVee, Plex
Other services may have some free content available
2.2.4.8 Brand Results
At time you will see Brand Results such as the channel page for a streaming service. These results should be rated based on relevance to the query.
[dramas on prime video] → The Prime Video brand page would be very relevant and considered an “Excellent” result
[soccer] → MLS Season Pass page and MLS team pages would be very relevant and “Excellent” results
[sports] →
ESPN and MLS Season Pass pages are very relevant and “Excellent” results
Prime Video and Peacock both sometimes feature live sports but it is not as much of a focus for them and would be “Good” results
3 Rating Scale & Examples
Each of our three query types have different ways of applying the principles outline above. Reference the section below to see a deeper dive and examples on how to rate each type of query.
3.1 Browse
For browse queries there is no “Perfect” results. This is because, by their nature, the user is not looking for anything specific and is instead browsing. As a result the best rating is “Excellent” and the ratings are determined by relevance / connection to the query, popularity and recency.
General Guideline:
Excellent → The returned content is relevant to the query (not only by title), popular and recent
Good → The returned content is relevant to the query, and either popular or recent.
Acceptable → The returned content is relevant to the query and neither popular nor recent.
Off-Topic → Regardless of popularity and recency, if it is unlikely that the user would use the query to search for the returned content or there is no relationship between the query and the content.
3.1.1 Browse Examples
Browse Scale
Browse Examples
3.2 Similarity
For similarity queries we have a very different way of evaluating the results. Instead of looking at popularity and recency we are focused on how similar the content is to the referenced content. The primary intent of similarity searches and ratings is to capture “if a user liked piece of content x, will they also like piece of content y?”
Evaluate Similarity between the intended content and the returned content based on the following aspects:
Target Audience
Genre
Age Rating (Kids (PG) vs Everyone (PG-13, TV-14) vs Adults (R))
Factual Aspects
Cast & Crew: Actors, Producers, Studio, etc.
Setting: Location AND Time Period in which the content plays
Theme
What is the content about?
For similarity queries there may be additional aspects highlighted such as “movies like x”, “comedies similar to y”. In those cases the ratings should take the highlighted aspect as necessary to reach the minimum acceptable rating.
General Guideline:
Excellent → Similarity in Target Audience, Factual Aspect and Theme
Good → Similarity in 2 of the three categories (Target Audience, Factual Aspect and Theme) or just a close match in Target Audience
Acceptable → Similarity in only Factual Aspects, Theme or loose Target Audience match
Off-Topic → No Similarity
3.2.1 Similarity Examples
Similarity Scale
similarity examples
3.3 Navigational
Navigational queries are generally looking for a specific piece of content (i.e. a specific movie or show) or a finite set of content such as a film-trilogy. In these cases we allow for a “Perfect” rating for those specifically intended pieces of content.
Note: Navigational queries can also also receive results due to similarity, in that case refer to the similarity guidelines to rate the result.
[movies with the squirel and acorn] → Looking for Ice Age movie franchise, if something not from the franchise shows up it is rated on similarity
General Guideline:
Perfect → If navigational content and result is certainly primary intent of user, rate it perfect, regardless of popularity and recency. For Franchise related queries, please follow the Franchise specific guidelines as outlined above.
Excellent → If the returned content is a sequel or prequel of the intended content, or the content is part of a movie bundle with the intended content, rate the result as excellent.
Good → If the returned content is relevant to the query, and either recent or popular, or can be considered a secondary intent, rate it as good.
Acceptable → If the returned content is relevant to the query, but neither recent nor popular or can only poorly satisfy a secondary intent, it can be rated as acceptable.
Off-Topic → If the content is not relevant to the query or it is very unlikely that a user would search for the returned content with the given query, it can be rated off-topic.
3.3.1 Navigational Examples
Navigational Scale
Navigational Examples
3.3.3 Similarity Results on Navigational Queries
For similarity results on navigational queries use the above scale, but lower every rating by 1 (i.e. an “Excellent” becomes “Good”)
Below are the same examples from above, but showing how the similar result might show up on a navigational query. Navigational Similarity Scale
Navigational Similarity Examples
3.4 Special Cases
There are a variety of special cases to be aware of, where the rating scales are slightly different from what is written above. These scenarios will primarily happen within the Browse and Navigational categories.
Please note that the below guidelines are meant to augment the guidelines above, not override them. For instance the additional guidance for a query referencing a person has maximum rating of “excellent”, however, for a navigational query such as “movie where tom hanks is stuck on a island” the intended movie of “Cast Away” would still get a perfect rating.
3.4.1 Time Period
For queries that specify at time period of release use the following guidelines as recency is not an important aspect anymore.
Excellent:
Content which won major award(s) in the given year/decade
Ultra Popular (Approximately):
Top 50 most viewed shows/films associatedwith decade
Top 10 most viewed films associated with year
Ultra popular show has >=50% of seasons/episodes in decade
Good:
Popular shows/movie associated with year/decade
Show has 3+ seasons (or >50%) in decade
Popular (Approximately):
Top 100 most viewed in decade
Top 30 most viewed films from year
Acceptable:
Content released in year/decade
Show has 1+ season in decade
Off-Topic:
Content not from year/decade
.
Time Period Examples
3.4.2 Character
Intent for “Character” queries can be both Movies and TV Shows. Often, the “Franchise Ratings” (Appendix) will apply to ratings for characters. In cases where the main character name and the title of the content match, character shall be the dominating classification.
General Guideline:
Perfect:
The most popular and recent content featuring this character in a major role. (Apply “Franchise Query” Rule).
If only one content with the character is produced, this content can be rated perfect
Excellent:
Sequels/prequels for the show in which the character is best known
Other high-quality content featuring the character
Person page for well known actor/actress who plays the character
Good:
Content which is in the same franchise but about a different character
If the character has their own spinoff, the ‘parent’ show where the character first appeared
Acceptable:
Show/movie features the character in insignificant roleCharacter Scale
Character Examples
3.4.3 Person
Some queries may reference a person, be it an actor, director, producer, musician, etc. Use the guidelines below modify your rating for queries with such a reference. This guidance is primarily for Browse queries featuring a person.
General Guideline:
Excellent:
Content with the intended person as lead in cast & crew
Most popular documentary about the person (more than 1 possible if equal in popularity/quality)
Recent and popular live event with person
Content where person is a significant guest star. Hosted by reputable content creator.
Set of most popular content inspired by the person
Person page
Good:
Documentary about the person that is popular, but not most popular or recent
Popular live event with person
Popular content inspired by the person
Content with the intended person as cast & crew that is popular or recent
Acceptable
Unpopular content about/with the person
Content with the intended person as cast & crew (not as lead)
Person Scale
Person Examples
3.4.4 Awards
Users searching for an award show are generally interested in (1) watching nominated movies/shows before the award event, or (2) watching movies/shows which won the most recent award event. Recent winners should receive higher ratings, unless the query specifies a specific edition of the award event.
Definitions:
If award event upcoming: Nominations have already been announced for the next upcoming award event
If no award event upcoming: Nominations for next upcoming award event are not announced yet.
General Guideline:
Perfect:
Most recent award event show
Currently ongoing or upcoming live award event show
Excellent:
If award event upcoming: nominees (movies, tv shows or actors) for the upcoming award event. The returned content itself needs to be nominated
if movie returned, movie needs to be nominated
if artist page is returned, artist needs to be nominated
If no award event upcoming: winners of the most recent award event
Good:
If award event upcoming: winners of the most recent award event
If no award event upcoming: winners of previous award events, nominees of most recent award event
Similar award event shows
Acceptable:
Movies/Shows featuring people associated with award show
Nominees of previous award events
Exceptions:
If a year is specified in the query, consider only results relevant to the year
Perfect → content that won an award that year
Excellent → content that was nominated for that year
Off-Topic → content has no association with the award in the given year
Awards Scale
Awards Examples
3.4.5 Mentioned Content
Users may search for content by referencing another piece of content. In these cases
Typical Structure for Mentioned Content Queries:
Similarity: “shows like game of thrones”
'game of thrones' is the mentioned content here, per the franchise guidance below this does not include 'House of the Dragon'
Recursion: “movies with the actor of ron weasly”
'ron weasly' and by extension any Harry Potter movies that feature Ron Weasley is the mentioned content in this case
General Guideline:
Excellent:
Other highly relevant / related content
Acceptable:
The mentioned content(s)
Special Cases:
Franchise
Content under the same branding as the mentioned content is acceptable
Franchise content under different branding is rated according to the franchise / similarity rule
mentioned content examples
3.4.6 Live Event
Note that the examples below may have already taken place in the past. Please assume that live assets are either ongoing or upcoming.
When considering a sporting events popularity please consider multiple aspects that may affect popularity, here are some examples of factors:
Event: Some sports are very popular in the Olympics and not so much otherwise
League: An english user is likely more interested in the Premier League or Champions League than Ligue 1 in France
Level: The highest level professional sports are often more popular than lower leagues or college sports
Competition: Preseason is generally less popular than regular season events and playoffs / championship events are often the most popular
Note that this popularity is scaled based on the possible intents left, if the query points directly to a niche sport or event then we should not demote results because the specific sport/event is not popular overall, however, in broader live event queries and queries where live events are a potential secondary intent lower popularity live events should receive correspondingly lower ratings. Live Event Signifier General Guideline:
Perfect → ongoing or upcoming live event when query is for a team or single day event
Excellent → popular live events when query is for a sport, league or multi-day event. Also video on demand assets for recent championship events.
Good → less popular ongoing or upcoming live events related to the query that satisfy a secondary intent. Also movies and tv shows related to the query that is popular and recent
Acceptable → video on demand content for relevant non-championship live events. Also movies and tv shows related to the query that are not popular and recent
Live Event Scale
Live Event Examples
3.4.7 Non-Content Queries & Metadata Issues
You may come across queries such as [cancel subscription], [settings], or [log out] which are not relevant to any video content. For such queries, please use Problem. Please use with caution and explain your judgement in a comment.
You may come across a preview page which has incorrect or missing data such as wrong Cast and Crew, wrong Release Date, or the wrong artwork. For such issues, please use the rating Problem and describe the issue in the comment section Note: Release dates often vary between different locales and can be difficult to fully confirm, do not use Problem for release dates that are close to what is found through query research and instead only utilize the rating for significant deviations that are 10 or more years off. .
Comments
Post a Comment