Mastering Cues, Contexts, and Reward Schedules

Am I talking about making Wally an actor and a gambler? Well, yes and no. While he'll never memorize a script in a play (unless it's barking, maybe) or be able to play Texas Hold 'em or Blackjack for me (or even help me count cards), what I am talking about are principles that are similar to both.

First, we'll start with the "actor" side. Actors often rely on cues to help them remember what to say or do next. Along the same lines are stage directions, helping the actors move about at the right time to make the scene work.

The cues a dog gets are similar. They trigger him to action. However, there's more than one way dogs get cues and sometimes that gets lost. Here are ways I've seen Wally "take his cues" on what to do next.

• Socially - Based on interaction with another creature, be it me, another dog, or the kitten that's curious and decides to venture a sniff of Wally.
• Human Directed - This is the usual way cues are considered, us giving some kind of signal to the dog to do an action. Typically voice or hand signals (or both).
• Emotionally - The dog's emotional state can trigger an action. This action can then be rewarded, ignored, or even positively punished, though that's not advisable with emotions involved.
• Environmentally - The dog see something in the environment and takes an action as a result. Prey drive can be an example of this. Running an agility course requires some measure of this as well. Events that triggers action (picking up the leash makes the dog go to the door and wait) are environmental from the dog's perspective.

Some lament the fact that dogs are poor generalizers. I prefer to take advantage of it if I can. Some examples:

• Wally waits at some doors/entries in the house, but not all of them. I wouldn't want him waiting to go into the hallway, but I don't want him bolting out the front door. The "scenes" of the front door and the hallway are different, so he can learn to take different actions in those "scenes".
• Grass is different than sidewalk. The feel of the "terrain" on his paws became an environmental cue. On the grass, he's more free to explore as it's where he pees, more safe to roam, etc. But on the sidewalk, he's to stay close to me.
• Stopping in the middle of the field is different than stopping at the intersection. He's learned that he's not to go on that "black stuff" (the street) on his own and that when we stop in front of the "black stuff" he's to sit and wait.
• Food on the floor is different than food in the bowl. He won't jump on food that falls on the floor. He will stare at it like he wants it - as if he's trying to will it into his mouth. However, that same food in the food bowl, and he jumps it. That's fine - I don't want him to think he can't eat out of his bowl while also not wanting to him just suck up anything on the floor (could be onions, a grape seed, some bleach, etc.)
  

The dog doesn't necessarily take just one of the above cues to predict a behavior. In fact, dogs learn in context. What's a context? It's what the dog sees, feels, hears, and probably even smells in a given situation. It's the environment + you + dog's emotional state.

If ANY of those variables are different, the dog will interpret the situation as a new context. This is why it is said that dogs are poor at generalization. This means that every slight change is noticed and can change the behavior as a result. Cues that were learned in the living room suddenly "disappear" when out in the field. Everything is fine until a sound triggers anxiety and he can't "remember" his behaviors. These are examples of what Wally did when first learning behaviors. Working with his fearful nature really underscored the importance of emotion and situation to his behavior.

The good news is that you CAN generalize behaviors with dogs, but it requires consistency of at least a couple of the above cue sources. For example, since I don't want Wally bolting to God-knows-where (actually, he'll run home if he knows how to get there, but you get my point) when he's scared off-leash, I gave him an action to take every time he's feeling fearful. He's to recall (come) to my position and sit. Every time. This required consistency (and patience, and major treats) on my part. The fearful emotion would be consistent (of course) so the two combined can trigger him to the behavior.

Sometimes it's a two-edged sword, this casino mentality, and it comes when there are behaviors you actually DO want the dog to delete. The problem is, sometimes he gets the prize and that just makes him think "well I just have to keep trying it, and I will get rewarded!"

This is where the "self-rewarding" behaviors can be a problem. Nine times, you keep him from stealing the food, but the tenth time you get distracted by the kids, or the phone, or tree that fell on your car and the dog gets the food. Well, if the tree fell on your car, you probably don't care the dog got the food, but in the dog's mind, he got the prize. So instead of him starting to think, "I don't need to even bother trying." He gets the prize and now is like, "well, if it takes 10 times, maybe next time it might take 15 so I better keep at it," and the behavior becomes more reinforced.

This can also happen if you "give in". Say you don't want your dog to beg at the dinner table. For a few days, you stop giving in to his begging. The next day, he really wants what you're eating. You see those sad eyes, maybe even a whimper or a paw gently (or not so gently) touching your leg. You say "okay, just this once, but no more this week!" Well, the dog doesn't care about the rest of the week. He got the prize for his begging, so he'll keep it up, hoping that if he tries hard enough, he'll get the prize again.

The Downside with a Silver Lining

However, there's one situation where you want to do this, and that's shaping. By "giving in" and then making the dog exert "extra" effort, you can generate new behaviors. However, the main difference between shaping and the downside examples is that shaping isn't about causing extinction. It's about pushing the dog "to the edge" of extinction to get him to do "something extra" on the behavior, which is often a new behavior altogether.

That said, shaping is a powerful way to reinforce even known behaviors. Many times, my shaping sessions with Wally are just him offering me known behaviors in an effort to get me to reward him. He might sit to start with. C/T. Then he'll bark. C/T. He might bark, and I do nothing. He barks again. C/T. He barks. C/T. Another bark. Nothing. Two more barks. Nothing. At this point, if he offers a different behavior, he'll get a c/t. If he barks, maybe I will, maybe I won't (I probably would on either the 4th or 5th consecutive bark). So he learns to either "keep trying and don't give up" or "if it isn't working, try something else!" both of which are lessons I want him to keep forever.

Ah yes. Well, what's the main characteristic of slot machines? They're random. Sometimes you get "paid" sometimes you don't. Contrast that to, say, a lamp. Normally, it's flip/turn/push swtich, light comes on. Every time any time. It's so constant that you could bet on it and win a vast majority of the time.

So what's this got to do with dog training? Well, it's a topic called variable schedule of reinforcement but that's just a fancy phrase for random treat giving.

The power of this variable schedule of reinforcement, is that it works like the slot machine's "ability" to keep people trying "one more time". Sometimes, people win at the slots. If you take a learned behavior and work this into the situation, you can get the same kind of reaction from the dog.

What this does is help stave off extinction. Extinction is the dog deleting that behavior chain from his list of behaviors. If a behavior goes unrewarded long enough, then the dog will think the behavior is no longer wanted/worth doing. But, if you occassionally reward the behavior, then the dog will think "well if I keep doing it, maybe THIS time I'll get rewarded!" That's the same mentality as with the slots player as he/she puts token after token in.

You want your dog to keep putting in those "tokens" (i.e. his behaviors) in hopes of getting the reward, and if he does it quite well, the occassional jackpot. (see how this continues to tie in with gambling at the slots? Sometimes the player gets a small reward, sometimes NO reward, sometimes...JACKPOT!)

Now there's one difference in Slots for Dog Training: The dog doesn't know he always has a chance to win, at least not yet. In the beginning, always reward every success (and partial successes if you're using the shaping method). You want the dog to believe, "Yes, yes I can win this game!" Think of it as "suckering him in" (engaging his drive at you) so you can "keep him at the table" (keep him trying/performing) and giving you his "money" (behaviors).

Now, over time, the dog WILL remember all these rewards and "wins" and "prizes" he's earned when playing at your "casino". Every time you open for business, he'll be there, wanting to play and seeing if he can earn some more prizes from you. This is the same effect of players going to a real casino - they already know they have the chance of winning, the thrill is in trying to win. Over time, this will be the same reaction from your dog when it's time to train.

With the next hub, I'll go more into the games I play with Wally involving cues. Also more into how our shaping sessions go.

Hope to see you there!