Claims of US patent 6,307,576 and 6,611,278 in the name of Maury Rosenfeld were found not to be] patent eligible subject matter under 35 USC 101 because the novel features of the claim are overly abstract.
- Mere recitation of a generic computer cannot transform a patent –ineligible abstract idea into a patent-eligible invention to use in the same field.
- Avoid describing prior art in the specification.
- Clamining novel portions of an invention too broadly is dangerous.
- When a claim recites tangible steps, but the only new part of the claim is an abstract idea, that may contribute a claim to an abstract idea.
- Claiming “rules” may be seen as overly broad and possibly abstract.
Broad description of the invention
The patents claimmethods and systems to automatically create lip synchronisation and facial expression in three dimensional animated characters.
`576 Patent claim 1:
A method for automatically animating lip synchronization and facial expression of three-dimensional characters comprising:
obtaining a first set of rules that define output morph weight set stream as a function of phoneme sequence and time of said phoneme sequence;
obtaining a timed data file of phonemes having a plurality of sub-sequences;
generating an intermediate stream of output morph weight sets and a plurality of transition parameters between two adjacent morph weight sets by evaluating said plurality of sub-sequences against said first set of rules;
generating a final stream of output morph weight sets at a desired frame rate from said intermediate stream of output morph weight sets and said plurality of transition parameters; and
applying said final stream of output morph weight sets to a sequence of animated characters to produce lip synchronization and facial expression control of said animated characters.
Both patents are titled “Method for Automatically Animating Lip Synchronization and Facial Expression of Animated Characters.”
The patents explain that prior methods of animating lip synchronization and facial expressions were laborious and uneconomical. `576 Patent 1:14-31. The patents address that problem with an automated method of using “weighted morph targets and time aligned phonetic transcriptions of recorded text, and other time aligned data.” `576 Patent 2:64-3:12. The patents explain that in the relevant art, “`phonemes [are] defined as the smallest unit of speech, and correspond to a single sound.” `576 Patent 1:34-36. A sound recording can be transcribed into a “time aligned phonetic transcription” in which the timing of each phoneme is noted. `576 Patent 1:32-34. Such transcriptions can be created by hand or by automatic speech recognition programs. `576 Patent 1:39-43.
The patents explain that the prior art practice for 3-D computer generated speech animation was by manual techniques using a “morph target” approach. `576 Patent 1:44-46. That approach uses a reference model of a neutral mouth position in conjunction with “morph targets,” which are models of the mouth in non-neutral positions corresponding to different phonemes. `576 Patent 1:46-49. The reference model and morph targets all share the same “topology” of the mouth, defined by the same number and placement of “vertices” that designate specific points on the mouth. For example, vertex “n” on the neutral mouth and all of the morph targets may represent the left corner of the mouth. `576 Patent 1:51-54.
The “deltas,” or changes, of each vertex on each morph target relative to the corresponding vertex on the neutral model are computed as a vector to produce an individual “delta set” of vectors for each morph target. `576 Patent 1:58-62. From the neutral model, the animator need not move the mouth position all the way to a morph target. Instead, the animator can apply a value between 0 and 1, called the “morph weight,” to a delta set to move the mouth just a percentage of the way to the corresponding morph target. `576 Patent 1:63-2:1. For example, if the sound (morph target) is “oh,” and the morph weight is 0.5, the mouth only moves halfway between the neutral position and the “oh” morph target. `576 Patent 2:16-22. It is also possible to blend the morph targets, for example, 0.3 “oh” and 0.7 “ee,” resulting in a mouth position exhibiting a combination of the “oh” and “ee” sound characteristics. `576 Patent 2:23-28.
According to the patents, applying the appropriate morph weights in the prior art was usually done using a “keyframe” approach. In the keyframe approach, an artist sets the morph weights at certain important times, and a computer program then interpolates each of the channels at each frame between the keyframes. `576 Patent 2:29-34. The patents state that this method requires the artist to manually set a large number of keyframes, which is tedious, time consuming, and inaccurate. `576 Patent 2:34-37. Therefore, an object of the invention is to provide “an extremely rapid and cost effective means to automatically create lip synchronization and facial expression in three dimensional animated characters.” `576 Patent 2:50-54.
The invention “utilizes a set of rules that determine the system[‘]s output comprising a stream or streams of morph weight sets when a sequence of timed phonemes or other timed data is encountered.” `576 Patent 3:3-7. The invention includes:
[C]onfiguring a set of default correspondence rules between a plurality of visual phoneme groups and a plurality of morph weight sets; and specifying a plurality of morph weight set transition rules for specifying durational data for the generation of transitionary curves between the plurality of morph weight sets, allowing for the production of a stream of specified morph weight sets to be processed by a computer animation system . . . .
`576 Patent 3:23-30.
Defendants argue that the claims of both patents in suit are patent ineligible under 35 U.S.C. § 101 because they merely “set forth the previously-known animation method as a series of mathematical steps, and instruct the user to perform those steps on a computer.” Mot., Docket No. 338 at 12.
35 U.S.C. § 101 “defines the subject matter that may be patented under the Patent Act.” Bilski v. Kappos, 561 U.S. 593, ___, 130 S.Ct. 3218, 3225 (2010). It provides:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Id. “In choosing such expansive terms . . . modified by the comprehensive `any,’ Congress plainly contemplated that the patent laws would be given wide scope” “to ensure that `ingenuity should receive a liberal encouragement.'” Id. (quoting Diamond v. Chakrabarty, 447 U.S. 303, 308 (quoting 5 Writings of Thomas Jefferson 75-76 (H. Washington ed. 1871)) (some internal quotation marks omitted).
The “wide scope” of patent eligibility is not unlimited. Instead, the Supreme Court has invented or discovered “three specific exceptions to § 101’s broad patent-eligibility principles: `laws of nature, physical phenomena, and abstract ideas.'” Bilski, 130 S.Ct. at 3225 (quoting Chakrabarty, 447 U.S. at 309).
In Mayo Collaborative Services v. Prometheus Laboratories, Inc., 132 S.Ct. 1289 (2012), the Supreme Court “set forth a framework for distinguishing patents that claim laws of nature, natural phenomena, and abstract ideas from those that claim patent-eligible applications of those concepts.” Alice Corp. Pty. Ltd. v. CLS Bank Int’l, 134 S. Ct. 2347, 2355 (2014). That framework is as follows:
First, we determine whether the claims at issue are directed to one of those patent-ineligible concepts. If so, we then ask, “[w]hat else is there in the claims before us?” To answer that question, we consider the elements of each claim both individually and “as an ordered combination” to determine whether the additional elements “transform the nature of the claim” into a patent-eligible application. We have described step two of this analysis as a search for an “`inventive concept'” — i.e., an element or combination of elements that is “sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the [ineligible concept] itself.”
Id. at 2355 (citations omitted).
But despite its narrow holding, Alice did categorically establish a clear rule that had previously been subject to debate: “mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.” 134 S.Ct. at 2358.
And, while the boundaries of the judicial exceptions remain subject to further development, the Supreme Court has clearly stated the policy underlying those exceptions, i.e. avoiding patents that “too broadly preempt the use of a natural law [or abstract idea].” Mayo, 132 S.Ct. at 1294. Thus, patent law should “not inhibit further discovery by improperly tying up the future use of laws of nature [or abstract ideas].” Id. at 1301.
Mayo discussed the Supreme Court’s 1854 decision upholding many of Samuel Morse’s telegraph patent claims, but invalidating the most general claim, which covered “the use of the motive power of the electric or galvanic current . . . however developed, for making or printing intelligible characters, letters, or signs, at any distances.” Id. The Supreme Court presciently explained that such a claim would inhibit, rather than promote, the progress of the useful arts:
For aught that we now know some future inventor, in the onward march of science, may discover a mode of writing or printing at a distance by means of the electric or galvanic current, without using any part of the process or combination set forth in the plaintiff’s specification. His invention may be less complicated — less liable to get out of order — less expensive in construction, and in its operation. But yet if it is covered by this patent the inventor could not use it, nor the public have the benefit of it without the permission of this patentee.
Id. (quoting O’Reilly v. Morse, 15 How. 62, 113 (1854).) True, patents always present some impediment to follow-on innovation. The principle is one of balance: patents should not “foreclose more future invention than the underlying discovery could reasonably justify.” Mayo, 132 S.Ct. at 1301.
Of course, § 101 is not the sole, or even primary, tool to ensure that balance. Every condition of patentability set forth in the Patent Act acts to ensure that patents promote, rather than retard, the progress of science and useful arts. For example, in a manner quite similar to recent § 101 jurisprudence, “[t]he written description requirement guards against claims that `merely recite a description of the problem to be solved while claiming all solutions to it and . . . cover any compound later actually invented and determined to fall within the claim’s functional boundaries.'” Abbvie Deutschland GmbH & Co., KG v. Janssen Biotech, Inc., ___ F.3d ___, XXXX-XXXX, 2013 WL 2937477, 11 (Fed. Cir. July 1, 2014) (quoting Ariad Pharm., Inc. v. Eli Lilly & Co., 598 F.3d 1336, 1353 (Fed. Cir. 2010)).
However, scholars have argued that the written description and enablement doctrines of § 112, as currently applied, do not adequately prevent unwarranted obstructions to follow-on innovation, and have urged that § 101 can and should do so. See, e.g., Lemley et al., Life After Bilski, 63 Stan. L. Rev. 1315, 1330 (2011) (cited in Mayo, 132 S.Ct. at 1301-03, 1304); but see Lemley, Point of Novelty, 105 Nw. U. L. Rev. 1253, 1279 (2011) (“[T]here is good reason to worry about overbroad patent claims that lock up a wide swath of potential future applications. But the enablement and written description doctrines largely address that concern.”).
In any event, the Supreme Court has spoken, and § 101 now plays an important limiting role. But District Courts and the Federal Circuit are now left with the task of figuring out when the “two-part” test is satisfied. Perhaps something like the function-way-result test used to evaluate infringement under the doctrine of equivalents might be useful. Thus, in one long-standing formulation, an accused instrumentality infringes “if it performs substantially the same function in substantially the same way to obtain the same result.” Union Paper-Bag Mach. Co. v. Murphy, 97 U.S. 120, 125 (1877); InTouch Technologies, Inc. v. VGO Commc’ns, Inc., 751 F.3d 1327, 1343 (Fed. Cir. 2014).
The test in practice often focuses on the “way” aspect of the test, because function and result are often identical in the patent and accused product, and the question is whether the accused infringer uses the same “way.” Laura A. Handley, Refining the Graver Tank Analysis with Hypothetical Claims: A Biotechnology Exemplar, 5 Harv. J.L. & Tech. 36 (1991) (“In practice, the second prong of the test — `substantially the same way’ is often emphasized, since most infringement suits result from competition for a given market niche which dictates the `function’ and `result’ prongs.”) (citing Perkin-Elmer Corp. v. Westinghouse Elec. Corp., 822 F.2d 1528, 1531 (Fed. Cir. 1987)).
Similarly, the question in the abstract idea context is whether there are other ways to use the abstract idea in the same field. If so, the Supreme Court has expressly encouraged others to find those other ways, without being held back by patents that preempt the whole concept. Mayo, 132 S.Ct. at 1294 (citing O’Reilly, 15 How. at 113); Alice, 134 S.Ct. at 3258 (noting “the pre-emption concern that undergirds our § 101 jurisprudence.”).
Concomitantly, we must be wary of facile arguments that a patent preempts all applications of an idea. It may often be easier for an infringer to argue that a patent fails § 101 than to figure out a different way to implement an idea, especially a way that is “less complicated — less liable to get out of order — less expensive in construction, and in its operation.” O’Reilly, 15 How. at 113. But the patent law does not privilege the leisure of an infringer over the labors of an inventor. Patents should not be casually discarded as failing § 101 just because the infringer would prefer to avoid the work required to develop non-infringing uses of the abstract idea at the heart of an appropriately circumscribed invention.
Defendants argue that the patents-in-suit are directed to a “fundamental, abstract animation practice,” namely, “the abstract idea of rules-based synchronization of animated mouth movement.” Mot., Docket No. 338 at 12. That is, Defendants argue that the patents cover the mere idea of using rules for three-dimensional lip synchronization, without requiring specific content for those rules. Id. at 12-13. But considered standing alone, the asserted claims do not seem to cover any and all use of rules for three-dimensional lip synchronization.
Facially, the claims do not seem directed to an abstract idea. They are tangible, each covering an approach to automated three-dimensional computer animation, which is a specific technological process. They do not claim a monopoly, as Defendants argue, on “the idea that the human mouth looks a certain way while speaking particular sounds,” “applied to the field of animation.” Mot., Docket No. 338 at 12, n.9. Further, the patents do not cover the prior art methods of computer assisted, but non-automated, lip synchronization for three-dimensional computer animation.
And according to Defendants, they do not cover the automated methods of lip synchronization for three-dimensional computer animation that Defendants employ. It is hard to show that an abstract idea has been preempted if there are noninfringing ways to use it in the same field. Section 101 motions can place parties in unfamiliar and uncomfortable positions: here it is to the patentee’s advantage to identify noninfringing alternatives, and it is the accused infringer’s advantage to posit the lack of any; the reverse of their positions at the infringement and damages stages of the case.
At first blush, it is therefore difficult to see how the claims might implicate the “basic underlying concern that these patents tie up too much future use of” any abstract idea they apply. Mayo, 132 S. Ct. at 1302; Alice, 134 S.Ct. at 2358 (noting “the pre-emption concern that undergirds our § 101 jurisprudence”).
However, for purposes of the § 101 analysis, it is not enough to view the claims in isolation. Instead, when determining whether a patent contains an adequate inventive concept, the Court must factor out conventional activity. That is because the inclusion of “well-understood, routine, conventional activity” previously used in the field “is normally not sufficient to transform an unpatentable law of nature [or abstract idea] into a patent-eligible application . . . .” Mayo, 132 S.Ct. at 1298. Further, in addition to evaluating each step of the claim, the claims must be considered as “an ordered combination.” Alice, 132 S.Ct. at 2355.
This dual analysis tracks the law’s long-standing concern with patents that consist of old material with the addition of a new, but abstract, idea: “the vice of a functional claim exists not only when a claim is `wholly’ functional, if that is ever true, but also when the inventor is painstaking when he recites what has already been seen, and then uses conveniently functional language at the exact point of novelty.” Gen. Elec. Co. v. Wabash Appliance Corp., 304 U.S. 364, 371, 58 S. Ct. 899, 903 (1938). An abstract idea is the extreme case of functional language.
Thus, where a claim recites tangible steps, but the only new part of the claim is an abstract idea, that may constitute a claim to an abstract idea. See Alice, 134 S. Ct. at 2358. (disregarding the presence of a computer in the claim given “the ubiquity of computers”); Mayo, 132 S.Ct. at 1297-98 (claim step calling for administration of drug only disregarded because it “refers to the relevant audience, namely doctors who treat patients with certain diseases with thiopurine drugs”; claim step of determining the level of the relevant metabolites disregarded because it was “well known in the art”).
Here, the patents teach that in the prior art, three-dimensional character lip synchronization was performed using a “timed data file of phonemes having a plurality of sub-sequences,” as recited in the claims. `576 Patent 1:32-43. But the prior art did not, according to the patents, involve obtaining rules that define output morph weight sets as a function of the phonemes, or using those rules to generate the morph weight sets. Instead, an artist manually set the morph weights at certain important keyframes, and a computer program then interpolated the frames between the keyframes. `576 Patent 2:29-37. Therefore, while tangible, the steps of (1) using a timed phoneme transcript, (2) setting morph weight sets at keyframes, or (3) interpolating between keyframes, are not “inventive steps” that could transform the claims herein into patent eligible subject matter, if those claims are directed to an abstract idea.
In attacking the claims as simply drawn to the abstract idea of “rules-based lip-synchronized animation on a computer,” Mot., Docket No. 338 at 3, Defendants’ argument does not account for the presence in the claims, or the Court’s construction, of “morph weight set.” The Court construed “morph weight set” as a “set of values, one for each delta set, that, when applied, transform the neutral model to some desired state, wherein each delta set is the [set of vectors] from each vertex on the neutral (reference) model to each vertex on a model of another mouth position.” Rulings on Claim Constr., Docket No. 298-1 at 9.
However, the patents themselves teach that the prior art includes using morph targets that correspond to phonemes and calculating delta sets that contain the vectors from each vertex on the neutral model to the morph target. `576 Patent at 1:44-62. So, while Defendant’s characterization is overly broad, it would be fair to characterize the claims as drawn to the idea of automated rules-based use of morph targets and delta sets for lip-synchronized three-dimensional animation. Indeed, Plaintiff’s expert opines that:
A central part of the creative insight of the patents is the realization to use the specific approach of using morph weight set representations of the facial shape coupled with rules, including explicit and distinct timing rules, to generate keyframes. This approach uniquely provides the automation required to produce animation in a cost-effective way, yet provided the necessary artistic control required to produce commercial grade animation.
Declaration of Michael Gleicher, Ph.D. in Supp. of Opp’n, Docket No. 345, ¶ 20. Defendants object to this testimony, because “[t]he Court may not consider declarations in opposition to a Rule 12(c) motion without converting the motion to a motion for summary judgment.” Defs.’ Objections to Declarations Filed in Connection with Motion for Judgment on the Pleadings, Docket No. 351 at 2. It is unclear how that response helps Defendants. Certainly, one option is for the Court to deny the Motion as presenting an issue that turns on the facts.
However, nothing in the Declaration affects the analysis. In the paragraph quoted above, Plaintiff’s expert opines that a central part of the patents is “using morph weight set representations of the facial shape coupled with rules, including explicit and distinct timing rules, to generate keyframes.” Everyone appears to agree with that characterization, except that Defendants point out that no particular “explicit and distinct” rules are required by the claims. The question is therefore whether the inclusion of that concept in the claims satisfies § 101 given (1) the prior art, and (2) the fact that the claims do not require any particular rules.
A consideration of the prior art recited in the patents shows that the point of novelty here is the idea of using rules, including timing rules, to automate the process of generating keyframes. The following chart compares the `576 Patent’s claim elements to the prior art described in that patent.
|Step||Admitted Prior Art|
|A method for automatically animating
lip synchronization and facial
expression of three-dimensional characters comprising:
|Automating the process is the focus of the invention. However, the patent teaches that in the prior art, the use of computerized interpolation partially automated the process by allowing animators to set mouth shapes only at keyframes, rather than at every frame, as would be the case in hand-drawn animation. `576 Patent 2:31-34.|
|obtaining a first set of rules that define output morph weight set stream as a function of phoneme sequence and time of said phoneme sequence.||Rules for defining morph weight sets as a function of phoneme sequence are disclosed as within the prior art. ‘576 Patent 1:44-2:28. Rules for defining morph weight sets as a function of timing are not; instead, the timing results from the artist’s choice of keyframes. `576 Patent 2:29-34. Note, however, that no particular timing rules are required by any claim.|
|generating an intermediate stream of output morph weight sets and a plurality of transition parameters between two adjacent morph weight sets by evaluating said plurality of sub-sequences against said first set of rules;||An intermediate stream of morph weight sets is disclosed as being part of the prior art through the keyframes manually set by the artist. `576 Patent 2:29-34. The transition parameters are not. Those parameters flow from the timing rules.|
|generating a final stream of output morph weight sets at a desired frame rate from said intermediate stream of output morph weight sets and said plurality of transition parameters; and||The patent teaches that the prior art generated the final stream by interpolating between the keyframes. `576 Patent 2:29-34. Again, transition parameters are not disclosed as being within the prior art.|
|applying said final stream of output
morph weight sets to a sequence of
animated characters to produce lip
synchronization and facial expression
control of said animated characters.
|Both the final set of output morph weight sets and applying those sets are covered by the interpolation process of the prior art. `576 Patent 2:29-34.|
So, what the claim adds to the prior art is the use of rules, rather than artists, to set the morph weights and transitions between phonemes. However, both of these concepts are specified at the highest level of generality. At the hearing on the Motion, Plaintiff emphasized that the rules inventively take into account the timing of the phoneme sequence. But the specification states clearly that “[i]n operation and use, the user must manually set up default correspondence rules” that “specify the durational information needed to generate appropriate transitionary curves between morph weight sets, such as transition start and end times.” `576 Patent 6:46-54. Thus, the user, not the patent, provides the rules. And while the patent does provide an example of a very partial set of default and secondary rules, it expressly states that “this is only an example of a set of rules which could be use[d] for illustrative purposes, and many other rules could be specified according to the method of the invention.” `576 Patent 7:36-9:23. Because the claim purports to cover all such rules, in light of the prior art, the claim merely states “an abstract idea while adding the words `apply it.'” Alice, 134 S. Ct. at 2358 (quoting Mayo, 132 S.Ct. at 1294) (some quotation marks omitted). The same is true for claim 1 of the `278 Patent, which does not differ in a manner relevant to this analysis.
Here, while the patents do not preempt the field of automatic lip synchronization for computer-generated 3D animation, they do preempt the field of such lip synchronization using a rules-based morph target approach. And if, as Plaintiff suggests, the inventive step is the use of timing rules, given the state of the prior art, that still leaves an abstract idea at the point of novelty, and preventing the development of any additional ways to use that abstract idea in the relevant field. See Alice, 134 S. Ct. at 2360 (“the claims at issue amount to `nothing significantly more’ than an instruction to apply the abstract idea of intermediated settlement using some unspecified, generic computer”).
Defendants argue that a “patentee simpl[y] does not waste the time, money and effort to prosecute a patent application for an invention they casually indicate was known in the art.” Opp’n, Docket No. 344 at 10-11. But a § 101 defect does not mean that the invention was in the prior art. The invention here may have been novel, but the claims are directed to an abstract idea. And the patent’s casual — and honest — description of the prior art was made at a time when, under the then-prevalent interpretation of the law, such admissions were unlikely to be harmful. One unintended consequence of Alice, and perhaps of this and other decisions to come, is an incentive for patent applicants to say as little as possible about the prior art in their applications.
Plaintiff points to one Defendant’s contemporaneous characterization of Plaintiff’s system as “revolutionary.” Opp’n, Docket No. 344 at 1 (quoting Decl. of John Petrsoric In Opp’n to Mot., Docket No. 346, Ex. 2, January 27, 1999 Warner Bros. Memorandum (inviting colleagues to a demonstration of Plaintiff’s “revolutionary lip synch technique” that “utilizes proprietary software.”)).
This argument is unpersuasive in this context for two reasons. First, for purposes of the § 101 inquiry, which is different from the § 103 inquiry, the revolutionary nature of an abstract idea does not weigh in favor of patentability. See Mayo, 132 S. Ct. at 1293 (“Einstein could not patent his celebrated law that E=mc2. . . . Such discoveries are `manifestations of . . . nature, free to all men and reserved exclusively to none.'”) (quoting Chakrabarty, 100 S.Ct. at 2204). Second, there has been no showing that the cited praise relates to the claims in all their breadth, rather than to a particular implementation that is not specified by the claims. Thus, the inventor’s specific implementation of the abstract idea represented by the claim may have been of significant value beyond that of the abstract idea itself.
This case illustrates the danger that exists when the novel portions of an invention are claimed too broadly. As noted above, the claims here are drafted to give the impression of tangibility, but the Supreme Court has “long warn[ed] . . . against interpreting § 101 in ways that make patent eligibility depend simply on the draftsman’s art. “Alice, 134 S. Ct. at 2351 (citing Mayo, 132 S.Ct. at 1294). When examined in light of the prior art, the claims are directed to an abstract idea, and lack an “inventive concept” “sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the [abstract idea] itself.” Id. at 2355 (citations omitted).