ScalaCheck

Property-Based testing

Qui suis-je ?

  • Benoît Lemoine
  • Développeur FullStack (Scala / TypeScript) chez Captain Dash
  • @benoit_lemoine

Parlons tests

C'est quoi un test ?

  • une procédure...
  • ...automatisé...
  • .. permettant de vérifier le bon fonctionnement d'un programme
Testing shows the presence, not the absence of bugs
Dijkstra.

Test unitaire

Tests basés sur l'exemple


"add should sum the numbers" in {
  add(1, 2) shouldBe 3
  add(123, 0) shouldBe 123
}

Implémentation respectant le test


  def add(x:Int, y:Int):Int = (x, y) match {
    case (1, 2) => 3
    case (123, 0) => 123
    case (_, _) => x
  }

ScalaCheck

  • Librairie Scala pour faire du test de propriété
  • Utilisable seule
  • Intégrée dans ScalaTest et Specs

Test de propriété


import org.scalacheck.Properties
import org.scalacheck.Prop.forAll

object AddSpecification extends Properties("add") {
  property("0 should be right neutral") = forAll { (x: Int) =>
    add(x, 0) == x
  }
}
          

            [info] + add.0 should be right neutral: OK, passed 100 tests.

Ce qu'il se passe en vrai


import org.scalacheck.Properties
import org.scalacheck.Prop.forAll
import org.scalacheck.Prop.collect

object AddSpecification extends Properties("add") {
  property("0 should be right neutral") = forAll { (x: Int) =>
    collect(x) {
      add(x, 0) == x
    }
  }
}

Ce qu'il se passe en vrai


[info] + add.0 should be right neutral: OK, passed 100 tests.
[info] > Collected test data:
[info] 12% 2147483647
[info] 12% -2147483648
[info] 10% 0
[info] 9% 1
[info] 6% -1
[info] 1% -1691614918
[info] 1% 670895397
...
[info] 1% 1334365593
[info] 1% 1694241903
[info] 1% 425822646
[info] 1% -97876626
[info] 1% -625637529

En cas d'erreur


object AddSpecification extends Properties("add") {
  property("0 should be left neutral") = forAll { (x: Int) =>
    add(0, x) == x
  }
}
          

[info] ! add.0 should be left neutral: Falsified after 0 passed tests.
[info] > ARG_0: 1
[info] > ARG_0_ORIGINAL: 256088374

Exemple Shrink


def pad(str:String, n:Int):String = if(str.length < n) {
  str.padTo(n, '0')
}  else {
  str
}

Exemple Shrink


property("padded str length should be 10") =forAll {(str: String) =>
   collect(str) {
     pad(str, 10).length == 10
   }
 }
          

[info] ! add.padded str length should be 10: Falsified after 17 passed tests.
[info] > ARG_0: "蒨噯穉洽ᄼ嫕ᛊ˕皑"
[info] > ARG_0_ORIGINAL: "蒨ꎪ뱏㒫ⵏ豙噯穉洽ᄼ嫕ᛊ˕皑"
[info] > Collected test data:
[info] 28%
[info] 6% 纞⥯
[info] 6% 蒨噯穉洽ᄼ嫕ᛊ˕皑
...
[info] 6% 횏㞤㡛
[info] 6% שּ繏Гဘ욖䇨ࠢ⇱
[info] 6% 䲫褐ተ鳟
[info] 6% 㪢癫
[info] 6% 獒꽺啼᯽鲞浡硝⤜̇
[info] 6% 洭ॣࡨ盽㶴

À quoi ça sert ?

  • Tests de propriété algébrique
  • Associativité myFn(x, myFn(y, z)) == myFn(myFn(x,y), z)
  • Commutativité myFn(x, y) == myFn(y, x)
  • Element neutre myFn(x, e) == x
  • Element absorbant myFn(x, e) == e
  • etc.

Exemple

List est un Functor


property("identity") = forAll {(list: List[Long]) =>
  list.map(identity) == list
}

property("composite") = forAll {(list: List[Long],
                                 fn1:Long => String,
                                 fn2:String => Int) =>
  list.map(fn1).map(fn2) == list.map(fn1 andThen fn2)
}

Exemple

de la vraie vie

  • deserialize(serialize(myObject)) == myObject
  • findById(save(entity)) == entity
  • compress(input).size <= input.size

Comment ca marche ?

Gen


val genInt:Gen[Int] = Gen.choose(10, 100)
forAll(genInt) { (x:Int) =>
  x > 9 && x < 101
}

Créer un Gen

  • Gen.numStr, Gen.alphaStr
  • Gen.choose(0, 1000)
  • Gen.oneOf("A", "E", "I", "O", "U")
  • Gen.option(myOtherGen)
  • Gen.listOf(myOtherGen)

Gen


val evenGen:Gen[Int] = Gen.choose(0, 1000).suchThat( _ % 2 == 0)
val anotherEvenGen:Gen[Int] = Gen.choose(0, 500).map(_ + _)

Gen est monadique


case class User(id:Int, name:String)


val genUser:Gen[User] = for {
  id <- Gen.choose(0, Int.MaxValue)
  name <- Gen.alphaStr.suchThat(_.length > 0)
} yield User(id, name)

forAll(genUser) { (user:User) =>
  user.name.length > 0
}

Les arbitrary


val genUser:Gen[User] = for {
  id <- Gen.choose(0, Int.MaxValue)
  name <- Gen.alphaStr.suchThat(_.length > 0)
} yield User(id, name)

implicit val arbUser:Arbitrary[User] = Arbitrary(genUser)

forAll { (user:User) =>
  user.name.length > 0
}

Exemple : java.time.LocalDate


val genLocalDate:Gen[LocalDate] = for {
  year <- Gen.choose(1980, 2040)
  month <- Gen.choose(1, 12)
  day <- Gen.choose(1, Month.of(month).length(Year.isLeap(year)))
} yield LocalDate.of(year, month, day)

Flemme et Gen de case class

https://github.com/alexarchambault/scalacheck-shapeless 

case class User(id:Int, firstName:String, lastName:String) {
  def name = firstName + " " + lastName
}

import org.scalacheck.Shapeless._
forAll((user:User) => {
  user.name.contains(user.firstName)
})

Flemme et Gen d'ADT

https://github.com/alexarchambault/scalacheck-shapeless 

sealed trait Planetoid
case object Mercury extends Planetoid
case object Venus extends Planetoid
case class JovianMoon(name:String) extends Planetoid

import org.scalacheck.Shapeless._
forAll((planetoid:Planetoid) => {
  planetoid.toString().length > 0
})

Attention !

Ne pas recoder la fonction dans le test


case class User(id:Int, firstName:String, lastName:String) {
  def name = firstName + " " + lastName
}

forAll((user:User) => {
  user.name == firstName + " " + lastName
})

Exemples du vrai monde


property("serialize and deserialize should be inverse") =
  forAll { (user: User) =>
    Json.toJson(user).asOpt[User] == Some(user)
  }

property("what's saved in database can be retrieved") =
  forAll { (user:User) =>
    User.save(user).map(_.id).flatMap(User.findById) == Some(user)
  }

Exemples du vrai monde


property("identity don't modify the functor") =
  forAll { (response:HttpReponse[String]) =>
    response.map(identity) == response
  }
property("functor is associative") =
  forAll { (response:HttpReponse[String],
            f1: String => Int,
            f2:Int => Long) =>
    response.map(f1).map(f2) == response.map(f1 andThen f2)
  }

Avec scalaz


implicit def respEqual[A]: Equal[HttpResponse[A]] = Equal.equalA
implicit val respFunctor = new Functor[HttpResponse] {
  def map[A, B](fa: HttpResponse[A])(f: A => B): HttpResponse[B] =
    fa.map(f)
}

functor.laws[HttpResponse].check

Dans les autres langages

  • Quickcheck (Haskell)
  • SwiftCheck (Swift)
  • JsVerify (JavaScript)
  • etc.

Bonus : Vérification de propriété par les types


          -- Idris
total plusZeroLeftNeutral : (right : Nat) -> 0 + right = right
plusZeroLeftNeutral right = Refl

total plusZeroRightNeutral : (left : Nat) -> left + 0 = left
plusZeroRightNeutral Z     = Refl
plusZeroRightNeutral (S n) =
  let inductiveHypothesis = plusZeroRightNeutral n in
    rewrite inductiveHypothesis in Refl

Questions ?